Building GenAI-based Simulation Environment
for End-to-End Autonomous Driving

Developing safe autonomous vehicles requires testing across an enormous range of driving conditions, yet physical road testing is fundamentally limited: it demands massive data collection efforts, is prohibitively expensive to scale, and cannot systematically surface long-tail safety-critical scenarios that are rare by definition. Traditional simulators partially address this gap but fall short in realism—failing to capture the sensor characteristics, agent behaviors, and environmental variability that modern, data-driven AV stacks actually encounter. As perception, prediction, and planning converge into unified end-to-end policies, the mismatch between scripted simulation and the true distribution of real-world driving has become a fundamental bottleneck to safe deployment.

This half-day tutorial bridges that gap through five focused talks. We open with a comprehensive overview of the autonomous driving and simulation landscape. We then introduce TeraSim—an open-source generative simulation framework designed to systematically uncover unknown unsafe events through realistic traffic modeling and intelligent scenario generation. Moving to scene fidelity, we examine NuRec, NVIDIA's neural reconstruction system for building high-quality, photorealistic simulation environments directly from real-world sensor data. Next, we explore how generative world models—exemplified by Cosmos—can synthesize diverse, controllable driving scenarios at scale using diffusion-based and video-foundation approaches. We close with long-horizon AV simulation through structured autoregressive world modeling, paired with retrieval-based evaluation for assessing AV policies over extended time horizons.

Participants will leave with both conceptual grounding and practical starting points. The tutorial highlights concrete open-source tools—including TeraSim, Cosmos, and NuRec—and traces a coherent pipeline from scene reconstruction through generative synthesis to safety-oriented evaluation. Whether you are a researcher designing new simulation methods or a practitioner building AV testing infrastructure, this tutorial offers a structured map of the generative simulation landscape and actionable entry points for your own work.