Get Started#

Welcome to FlashDreams! This page will guide you from a fresh checkout of the repository to running NVIDIA OmniDreams, the interactive HDMap-conditioned driving world-model demo built on FlashDreams.

Install#

FlashDreams uses the uv python package manager. Installation instructions for uv are available in the Astral documentation.

With uv installed, clone the repository and synchronize the OmniDreams workspace environment:

git clone https://github.com/NVIDIA/flashdreams.git
cd flashdreams
uv sync --package flashdreams-omnidreams --extra interactive-drive

This installs the FlashDreams core package, the OmniDreams integration, and the interactive-drive demo dependencies. The unified runner CLI is also available through the OmniDreams workspace member:

uv run --package flashdreams-omnidreams flashdreams-run --help

If you are developing FlashDreams itself or want every shipped runner in one environment, use uv sync --extra dev --extra runners instead.

Hugging Face authentication#

OmniDreams downloads demo scenes and checkpoints from Hugging Face the first time you run it. Before launching the demo, use a Hugging Face account with access to:

Then create a read token from Hugging Face token settings. The token must have read access to nvidia/omni-dreams-models and nvidia/omni-dreams-scenes. If you use a fine-grained token, include both repositories, plus nvidia/omni-dreams-samples if you plan to run the scripted example-data command. Export the token in the same shell where you run omnidreams-prepare, interactive-drive, or flashdreams-run:

export HF_TOKEN=hf_...

If the default cache under ~/.cache/huggingface is on a small disk, point HF_HOME at a larger volume before the first download:

export HF_HOME=/path/to/large/huggingface-cache

For more environment and container details, see the project README.

Run OmniDreams#

Start with the OmniDreams interactive driving demo. The command below runs the world model in one process and streams the generated camera view to a browser:

uv run --package flashdreams-omnidreams interactive-drive --stream-mjpeg :8080

On VRAM-constrained GPUs, add --offload-text-encoder to reduce peak VRAM usage by about 15 GB:

uv run --package flashdreams-omnidreams interactive-drive \
    --stream-mjpeg :8080 \
    --offload-text-encoder

This runs the text and first-frame encoders once per scene, frees them before building the diffusion pipeline, and reuses the cached embeddings across world-model resets. The trade-off is slower first loads and scene/variant switches.

Then open http://<server-ip>:8080/ in a browser on the same network. Use localhost when the browser is on the same machine. The default scene auto-stages on first launch when HF_TOKEN is set.

Optionally, pre-download the default scene and model assets so access errors and multi-GB downloads happen before the demo starts:

uv run --package flashdreams-omnidreams omnidreams-prepare \
    --scene-uuid clipgt-0d404ff7-2b66-498c-b047-1ed8cded60d4

The first launch can spend several minutes loading checkpoints, compiling kernels, capturing CUDA graphs, and autotuning. Later launches reuse cached assets and compiled kernels.

Scripted OmniDreams generation#

To generate a fixed OmniDreams rollout through flashdreams-run, use the single-view runner with bundled example data:

uv run --package flashdreams-omnidreams \
    flashdreams-run \
    omnidreams-sv-2steps-chunk2-loc6-lightvae-lighttae-perf \
    --example-data True \
    --example_data_uuid "239560dc-33d1-11ef-9720-00044bcbccac" \
    --total-blocks 20

See NVIDIA OmniDreams for local-window mode, WebRTC serving, scene variants, perf-manifest setup, multi-GPU launch commands, and additional example-data UUIDs.

Other recipes#

After OmniDreams, the model gallery covers the rest of the shipped FlashDreams recipes and their runner slugs. For example, the offline Self-Forcing Wan 2.1 1.3B T2V recipe runs with:

uv run --project integrations/self_forcing \
    flashdreams-run self-forcing-wan2.1-t2v-1.3b-taehv \
    --total-blocks 7

Additional setup options#

These steps are not required for the basic OmniDreams demo, but they are useful for development workflows and downstream projects.

Use FlashDreams as a library#

If you only need FlashDreams as a dependency in another project rather than running the shipped recipes, install it from PyPI instead:

pip install flashdreams

Or track the current main branch:

pip install "git+https://github.com/NVIDIA/flashdreams.git"

Where to next#

  • Models: every shipped recipe with its CLI slug, checkpoint source, and per-recipe knobs, alongside steady-state per-step latency numbers and reproducer commands.

  • NVIDIA OmniDreams: drive a world model in real time with the interactive-drive demo.

  • Inference pipeline overview: the generation loop end to end: KV cache, ring attention, CUDA-graph capture.

  • Config system: the configuration layer every recipe shares.

  • Add a new method: adding a new model or method as a plugin.

  • CLI and API Reference: Reference docs for the flashdreams-run CLI and the FlashDreams Python API.

  • Troubleshooting: common first-run failures and fixes.

Project and support#

FlashDreams is developed in the open on GitHub, and contributions are welcome. If you hit a bug or have a feature request, open an issue; if you have a fix or improvement, send a pull request. Browsing existing issues and pull requests is also a good way to see what others are working on.