Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[AAAI 2025] DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation
| Date | Stars |
|---|---|
| 2026-07-31 | 267 |
| 2026-08-04 | 269 |
| 2026-08-06 | 269 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<div align="center"> # DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation </div> Our team is actively working towards releasing the code for this project. We appreciate your patience and understanding as we navigate the necessary processes. Our new works, [DriveDreamer4D](https://drivedreamer4d.github.io/) and [ReconDreamer](https://recondreamer.github.io/), are released! ## [Project Page](https://drivedreamer2.github.io) | [Paper](https://arxiv.org/pdf/2403.06845.pdf) # Abstract World models have demonstrated superiority in autonomous driving, particularly in the generation of multi-view driving videos. However, significant challenges still exist in generating customized driving videos. In this paper, we propose DriveDreamer-2, which builds upon the framework of DriveDreamer and incorporates a Large Language Model (LLM) to generate user-defined driving videos. Specifically, an LLM interface is initially incorporated to convert a user's query into agent trajectories. Subsequently, a HDMap, adhering to traffic regulations, is generated based on the trajectories. Ultimately, we propose the Unified Multi-View Model to enhance temporal and spatial coherence in the generated driving videos. DriveDreamer-2 is the first world model to generate customized driving videos, it can generate uncommon driving videos (e.g., vehicles abruptly cut in) in a user-friendly manner. Besides, experimental results demonstrate that the generated videos enhance the training of driving perception methods (e.g., 3D detection and tracking). Furthermore, video generation quality of DriveDreamer-2 surpasses other state-of-the-art methods, showcasing FID and FVD scores of 11.2 and 55.7, representing relative improvements of 30% and 50%. <img width="919" alt="abs" src="https://github.com/f1yfisher/DriveDreamer2/assets/39218234/e23cf401-5943-4fb3-b0ed-7d183a9df5cd"> <img width="1327" alt="abs2" src="https://github.com/f1yfisher/DriveDreamer2/assets/39218234/edc11963-0443-4e3f-8309-8955330b4815"> # News - **[2024/12/18]** 🚀 Inference code and model weight for video generation are realsed! - **[2024/12/10]** 🎉 DriveDreamer-2 is accepted for AAAI'25!. - **[2024/03/11]** 🚀 We release the [DriveDreamer-2](https://drivedreamer2.github.io/) project! (Key features: multi-view video generation, user-friendly with LLM) # Getting Started Download model weights and preprocessing file [HERE](https://pan.baidu.com/s/1EPWcO_sCvlgqVFgNiDGk8w?pwd=dkjq). - [Installation](DOCS/install.md) - [Prepare Dataset & Env](DOCS/preparation.md) - [Train, Test, Visualization](DOCS/trainval.md) # Demo ## Results with Gnerated Structural Information **Daytime / rainy day / at night, a car abruptly cutting in from the right rear of ego-car.** <div align="center"> https://github.com/f1yfisher/DriveDreamer2/assets/39218234/0df78173-9dcd-42f4-8cf8-f7e16b724f82 </div> **Rainy day, car abruptly cutting in from the left rear of ego-car. (long video)** <div align="center"> https://github.com/f1yfisher/DriveDreamer2/assets/39218234/779fa0ad-595a-47f3-a52c-1c98c30fa640 </div> **Daytime, the ego-car changes lanes to the right side. (long video)** <div align="center"> https://github.com/f1yfisher/DriveDreamer2/assets/39218234/36c0f9e6-b9d1-4bd1-ab5c-f2c28eb3294c </div> **Rainy day, a person crosses the road in the front of the ego-car. (long video)** <div align="center"> https://github.com/f1yfisher/DriveDreamer2/assets/39218234/92f8cd31-a1b3-4516-ad03-331cf1ba4acb </div> ## Results with nuScenes Structural Information **Daytime / rainy day / at night, ego-car drives through urban street, surrounded by a flow of vehicles on both sides.** <div align="center"> https://github.com/f1yfisher/DriveDreamer2/assets/39218234/543656a4-729d-4b2c-b12d-6e75b3068669 </div> **Daytime / rainy day / at night, a bus is positioned to the left front of the ego-car, with a pedestrian near the bus.** <div align="center"> htt
Excerpt of 5,482 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:4ecae92bf8f5704f, llm:Repository description: 'DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation' (AAAI 2025). Python project for driving video generation using world models and LLM enhancements.
matched fp:4ecae92bf8f5704f, llm:Repository description: 'DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation' (AAAI 2025). Python project for driving video generation using world models and LLM enhancements.
matched fp:4ecae92bf8f5704f, llm:Repository description: 'DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation' (AAAI 2025). Python project for driving video generation using world models and LLM enhancements.