Cosmos-Framework

One Python package to train, post-train, evaluate and serve Cosmos3 omnimodal world models.

Quickstart Models GitHub

New: four world-simulation models: Transfer-Auto, Sim-Auto, Sim-Transfer and Sim-Bimanual.

Using a coding agent? Load the agent skills from a clone, or point it at llms.txt.

Choose your path

• Generate video, images, text and actions

  Text, images, video or actions in; images, video with optional audio, text or actions out.

  Run inference · Generation notebooks

• Simulate driving and robot scenes

  Multi-camera driving video from control videos; robot rollouts from recorded actions.

  Transfer-Auto · Sim-Bimanual

• Fine-tune on your data

  Supervised fine-tuning on JSONL or custom datasets, including robot policies.

  Choose a path · SFT recipes

• Improve with RL and preferences

  DPO, DiffusionNFT, GRPO, PPO and OPD, from a single GPU to a multinode Ray cluster.

  RL overview · RL quickstart

• Evaluate

  Score a checkpoint against ground truth, or reproduce PAI-Bench, Physics-IQ and R-Bench.

  Evaluate and score · PAI-Bench-G

• Deploy

  Serve with Ray and Gradio, vLLM, TensorRT-LLM or NIM, or stream robot actions from a policy server.

  Choose a serving path · Deploy with NIM