Agent skills¶
Cosmos-Framework ships skills for coding agents such as Claude Code and Codex: step-by-step instructions to install, run, post-train and troubleshoot Cosmos3 models, and to work on this repository. Each skill is a folder with a SKILL.md file; see Agent Skills.
Load the skills¶
The skills refer to files in the repository, so work in a clone:
git clone https://github.com/NVIDIA/cosmos-framework.git
cd cosmos-framework
- Codex loads the skills in
.agents/skills/when you start it in the clone. - Claude Code loads only the skills in
.claude/skills/, a subset of the table below. To load them while working in another project, start it withclaude --add-dir <path to the clone>. - For any skill your agent does not load, ask it to read the skill's
SKILL.md.
Agents reading this site can start from llms.txt. It links the Markdown of every page and every skill below.
Skills¶
| Skill | What it does | Folder |
|---|---|---|
| scenario-to-rds-hq | Convert a driving scenario or dataset in an unfamiliar format into RDS-HQ for WSM4 rendering, with final Hyperion camera retargeting by default. | .agents/skills/cookbooks/cosmos3/simulation/auto/wsm/skills/scenario-to-rds-hq/ |
| cosmos-automl | Optimize Cosmos Framework reasoner SFT hyperparameters or evaluation prompts with bounded AutoML. | .agents/skills/cosmos-automl/ |
| cosmos-framework-docs | Edit the Cosmos-Framework documentation and its MkDocs site. | .agents/skills/cosmos-framework-docs/ |
| cosmos3-codebase-nav | Navigate the Cosmos3 package codebase to find where parameters, configs, defaults, scripts, and documentation live. | .agents/skills/cosmos3-codebase-nav/ |
| cosmos3-env-troubleshoot | Diagnose and fix Cosmos3 environment, installation, and runtime errors. | .agents/skills/cosmos3-env-troubleshoot/ |
| cosmos3-inference-action-forward-dynamics | Run Cosmos3 action forward dynamics: a start image plus an action trajectory (AV ego poses, camera poses, DROID or UMI gripper actions, 57D human hand pose)… | .agents/skills/cosmos3-inference-action-forward-dynamics/ |
| cosmos3-inference-action-inverse-dynamics | Predict the action trajectory behind an observed video with the Cosmos3 action models (inverse dynamics): an autonomous-vehicle ego clip or a LeRobot robot… | .agents/skills/cosmos3-inference-action-inverse-dynamics/ |
| cosmos3-inference-action-policy | Run Cosmos3 action-policy inference for the DROID robot: one 640x540 concat_view observation (wrist + two exterior cameras) plus a task instruction in, an… | .agents/skills/cosmos3-inference-action-policy/ |
| cosmos3-inference-action | Entry skill of the Cosmos3 Action row: the action definition and embodiment table, the guardrail rule, the sample assets (AV, camera-pose, UMI, Bridge… | .agents/skills/cosmos3-inference-action/ |
| cosmos3-inference-generate-media | Generate images, videos, and audio-synchronized videos with the Cosmos3 audiovisual generator models (Edge / Nano / Super / Super-4Step) through any of the… | .agents/skills/cosmos3-inference-generate-media/ |
| cosmos3-inference-nim | Deploy, call and troubleshoot the Cosmos3 NIM (nvcr.io/nim/nvidia/cosmos3) as Generator or Reasoner from the public .agents/skills/cookbooks/cosmos3/nim… | .agents/skills/cosmos3-inference-nim/ |
| cosmos3-inference-reasoning | Run the Cosmos3 Reasoner (vision-language reasoning over images and video: captioning, VQA, temporal localization, 2D grounding, describe-anything, embodied… | .agents/skills/cosmos3-inference-reasoning/ |
| cosmos3-inference-transfer | Run Cosmos3 video transfer (control-to-video): a prompt.json caption plus one or more spatial control videos (edge/Canny, blur, depth, segmentation… | .agents/skills/cosmos3-inference-transfer/ |
| cosmos3-inference | Guide users through running Cosmos3 inference — offline batch generation, online serving with Ray and Gradio, parallelism options, input formats, sampling… | .agents/skills/cosmos3-inference/ |
| cosmos3-nano-sim-auto-inference | Prepare inputs or run Cosmos3-Nano-Sim-Auto camera-only or joint camera/LiDAR generation with a released or exported student in Cosmos Framework. | .agents/skills/cosmos3-nano-sim-auto/inference/ |
| cosmos3-nano-sim-auto-post-training | Distill Cosmos3-Nano-Transfer-Auto into Cosmos3-Nano-Sim-Auto using customer RGB, WSM, captions and LiDAR in Cosmos Framework. | .agents/skills/cosmos3-nano-sim-auto/post-training/ |
| cosmos3-nano-sim-auto-troubleshooting | Diagnose Cosmos3-Nano-Sim-Auto input, causal inference, distillation, checkpoint and export failures in Cosmos Framework. | .agents/skills/cosmos3-nano-sim-auto/troubleshooting/ |
| cosmos3-nano-sim-bimanual-inference | Generate action-conditioned video with Cosmos3 Nano Sim Bimanual in Cosmos Framework. | .agents/skills/cosmos3-nano-sim-bimanual/inference/ |
| cosmos3-nano-sim-bimanual-post-training | Train Cosmos3 Nano Sim Bimanual from Cosmos3-Nano with Cosmos Framework's four native stage TOMLs and a local training bundle. | .agents/skills/cosmos3-nano-sim-bimanual/post-training/ |
| cosmos3-nano-sim-bimanual-troubleshooting | Diagnose Cosmos3 Nano Sim Bimanual setup, data, training, checkpoint, inference, and evaluation failures by locating the first broken product contract. | .agents/skills/cosmos3-nano-sim-bimanual/troubleshooting/ |
| cosmos3-nano-sim-transfer-inference | Generate RGB video with a matched Cosmos3-Nano-Sim-Transfer checkpoint. | .agents/skills/cosmos3-nano-sim-transfer/inference/ |
| cosmos3-nano-sim-transfer-post-training | Prepare aligned spatial controls, RGB targets and captions for Cosmos3-Nano-Sim-Transfer; run native Cosmos Framework Stage 0–3 training with local assets… | .agents/skills/cosmos3-nano-sim-transfer/post-training/ |
| cosmos3-nano-sim-transfer-troubleshooting | Diagnose Cosmos3-Nano-Sim-Transfer setup, checkpoint loading, media alignment, training-state and output failures in an existing workflow. | .agents/skills/cosmos3-nano-sim-transfer/troubleshooting/ |
| cosmos3-nano-transfer-auto-inference | Prepare or run Cosmos3-Nano-Transfer-Auto in Cosmos Framework on camera WSM controls and per-view captions, with optional initial RGB and joint LiDAR. | .agents/skills/cosmos3-nano-transfer-auto/inference/ |
| cosmos3-nano-transfer-auto-post-training | Fine-tune the released Cosmos3-Nano-Transfer-Auto model on paired driving data in Cosmos Framework, or run the four-phase training curriculum from Cosmos3-Nano. | .agents/skills/cosmos3-nano-transfer-auto/post-training/ |
| cosmos3-nano-transfer-auto-troubleshooting | Diagnose cosmos3-nano-transfer-auto failures, including distorted side/rear views, WSM alignment, caption mismatch, unexpected frame counts, missing LiDAR… | .agents/skills/cosmos3-nano-transfer-auto/troubleshooting/ |
| cosmos3-post-training | Guide users through Cosmos3 supervised fine-tuning (SFT) post-training: preparing the example dataset and Wan2.2 VAE, converting the base checkpoint to DCP… | .agents/skills/cosmos3-post-training/ |
| cosmos3-quantization | Quantize Cosmos3 generator checkpoints to static-scale FP8 with the cosmos_framework.quantization recipe (NVIDIA TensorRT Model Optimizer): the Edge, Nano and… | .agents/skills/cosmos3-quantization/ |
| cosmos3-qwen3-vl-robovqa-sft | Fine-tune Qwen3-VL-8B-Instruct, a vision-language model from outside the Cosmos family, with the Cosmos Framework posttrain SFT path on robot-video question… | .agents/skills/cosmos3-qwen3-vl-robovqa-sft/ |
| cosmos3-setup | Guide users through Cosmos3 installation, environment setup, checkpoint downloading, and verification. | .agents/skills/cosmos3-setup/ |
| cosmos3-skill-authoring | Author new Cosmos3 agent skills and their eval configs, and get them through the skill eval. | .agents/skills/cosmos3-skill-authoring/ |