Waymo Builds a World Model to Simulate the “Impossible” on the Road

• Waymo unveiled the Waymo World Model, a generative simulation system built on Google DeepMind’s Genie 3 to create hyper-realistic 3D driving environments.
• The model produces high-fidelity camera and lidar outputs and allows engineers to control simulations through driving inputs, scene layouts, and language prompts.

Waymo has launched the Waymo World Model, a generative system designed to simulate driving environments at scale. Built on Genie 3, the model produces interactive 3D scenes tailored to autonomous driving. It generates both camera and lidar data and can recreate everyday traffic as well as rare events like tornadoes, floods, animals in the road, or wrong-way vehicles.

Unlike many simulation systems that rely only on recorded on-road data, the Waymo World Model can generate scenes beyond what the fleet has directly captured. Engineers can test alternative driving actions, adjust road layouts and traffic behavior, or use language prompts to change conditions like time of day and weather. The system demonstrates scenarios ranging from extreme weather and natural disasters to safety-critical events like wrong-way vehicles and reckless drivers, as well as unusual encounters with things like elephants, lions, and oversized tumbleweeds. It can also convert standard dashcam or mobile footage into multimodal simulations that reflect how the Waymo Driver would perceive the same scene.

In a blog post, the company said simulation is one of three key pillars of its approach to demonstrably safe AI, helping prepare the Waymo Driver for rare and complex situations before encountering them on public roads.

🌀 Tom’s Take:

Moving from replaying recorded drives to generating entirely new edge cases on demand gives Waymo a way to systematically stress-test rare and complex scenarios before they ever unfold on real roads.

Source: Waymo