Choose a post-training path¶
Choose your model and training signal together. Start with the model and algorithm support matrix for supported combinations, reference recipes and adaptation guidance. The native Cosmos3 recipes start from a matching Cosmos3 checkpoint; Posttrain also integrates Qwen, GR00T-N1.7 and π0.5.
| You have | Method | Where |
|---|---|---|
| Paired examples: prompt → video, video → text answer, observation → action | SFT | SFT |
| A teacher model you want to make faster (fewer steps) or causal | Distillation | Distillation |
| A reward function, verifier, simulator or preference pairs | RL | RL |
SFT¶
For Cosmos3, supervised fine-tuning runs on the native trainer. Each recipe is a TOML file under examples/toml/sft_config/; most have a paired examples/launch_sft_*.sh that runs the trainer under torchrun.
python -m cosmos_framework.scripts.train --sft-toml=examples/toml/sft_config/<recipe>.toml
- SFT recipes: Generator SFT for Cosmos3-Nano, Cosmos3-Super (LoRA) and Cosmos3-Edge, Reasoner SFT for all three, plus checkpoint conversion and export.
- Your own data: JSONL datasets and custom datasets.
- Robot actions from Cosmos3-Nano: DROID policy (reproduces Cosmos3-Nano-Policy-DROID), DROID forward dynamics and LIBERO-10 policy.
- Transfer-Auto: a four-phase RGB/LiDAR curriculum from Cosmos3-Nano, run with
cosmos_framework.scripts.train_experimentinstead. - Qwen text and supported VLM families: use the separate Posttrain conversation-SFT guide.
- Other compatible HF text/dense-VLM models can reuse the SFT core with tokenizer/template/media shaping and launcher wiring; see support and recipe availability.
- GR00T-N1.7: CF has an action-head flow-BC implementation but no public end-to-end offline SFT recipe. For customer demonstrations, start with Isaac-GR00T fine-tuning; see GR00T workflow boundaries.
Distillation¶
- Sim-Auto: three stages from Transfer-Auto, TF → TF-dCM → SF-DMD. To adapt Sim-Auto to your domain, adapt Transfer-Auto first and then distill, rather than running SFT on the released Sim-Auto checkpoint; this preserves its few-step generation.
- Sim-Transfer: a bidirectional Stage 0 teacher, then Stages 1–3: teacher forcing, four-step distillation and self-forcing.
- Sim-Bimanual: Stages 0–3 from Cosmos3-Nano. Check the training data before loading weights.
- Super four-step (DMD2): Cosmos3-Super-Text2Image or Cosmos3-Super-Image2Video as the teacher, the matching
-4Stepcheckpoint as the initial student. A six-iteration functional recipe, not a production reproduction.
RL¶
Use the RL and OPD recipe index to find a starting point for your model and method.
RL runs on the cosmos_framework.posttrain runtime in its own environment: the Framework base plus a locked dependency set. Start with the overview and the run guide.
- Generator: DPO learns from preference pairs collected in advance; DiffusionNFT generates and scores its own samples with a reward. The run-guide recipes start from Cosmos3-Nano.
- Reasoner: GRPO, PPO and OPD (on-policy distillation from a frozen teacher). The run-guide recipes use public Qwen models.
To start the Reasoner side from Cosmos3-Nano or Cosmos3-Super, export the checkpoint as a Qwen3-VL Hugging Face directory. For Cosmos3-Super, add --vlm-model-name Qwen/Qwen3-VL-32B-Instruct.
python -m cosmos_framework.scripts.convert_model_to_vlm_safetensors \
--checkpoint-path Cosmos3-Nano \
-o examples/checkpoints/Cosmos3-Nano-VLM
Joint PPO also trains GR00T-N1.7 and π0.5 robot policies on LIBERO-Spatial; see the GR00T-N1.7 and π0.5 guides. The runtime also has an SFT objective for Qwen text and vision-language models.
AutoML (preview)¶
AutoML searches Reasoner SFT hyperparameters against held-out evaluation; its GEPA Auto-Prompter optimizes evaluation prompts without training model weights. It requires the nvidia-automl-core wheel, which is not yet published to a package index. See AutoML.