Multiverse Transformer: 1st Place Solution for Waymo Open Sim Agents Challenge 2023
Abstract
This technical report presents our 1st place solution for the Waymo Open Sim Agents Challenge (WOSAC) 2023. Our proposed MultiVerse Transformer for Agent simulation (MVTA) effectively leverages transformer-based motion prediction approaches, and is tailored for closed-loop simulation of agents. In order to produce simulations with a high degree of realism, we design novel training and sampling methods, and implement a receding horizon prediction mechanism. In addition, we introduce a variable-length history aggregation method to mitigate the compounding error that can arise during closed-loop autoregressive execution. On the WOSAC, our MVTA and its enhanced version MVTE reach a realism meta-metric of 0.5091 and 0.5168, respectively, outperforming all the other methods on the leaderboard.
Presentation Slides
MultiVerse Transformer for Sim Agents
MultiVerse: Every decision a driver makes, at every fleeting moment, spawns a multitude of parallel universes.
MultiVerse: Our multiverse simulator generates a multitude of parallel universes. Each one mirrors the logged real-world driving data, yet diverges subtly in its own unique way.
Qualitative Results
Multi-modal behavior. A vehicle waiting to get onto the main road, the car turns left or right, or keeps waiting. ADV also demonstrates multi-modal behavior. Video is played at 2x speed.
Unprotected left turn. Vehicles paused at an intersection, ready to make an unprotected left turn, waiting for the oncoming traffic to clear.
Reactive agents. ADV makes a slow right-turn, forcing the agent behind it to slow down or stop.
Overtaking maneuver. A congested right lane, blocked by a large truck, and a free-flowing left lane. Trapped cars attempt to switch lanes, overtaking the ADV and the remaining slow traffic.
Reactive ADV and agents. Left: ADV slows down to allow a car merging onto the main road from a driveway. The car behind. Right: In the second simulation, ADV proceeds straight instead of waiting.
Paper
Yu Wang, Tiebiao Zhao, Fan YiMultiverse Transformer: 1st Place Solution for Waymo Open Sim Agents Challenge 2023
arXiv 2306.11868, 2023.
(PDF)