Skip to main content
Ctrl+K
CloudAI - Home CloudAI - Home

CloudAI

CloudAI - Home CloudAI - Home

CloudAI

Table of Contents

  • Overview
  • Get started
    • Install CloudAI
    • Run your first experiment
    • Understand experiment configuration
  • Run experiments
    • Set up your execution environment
    • Run on Slurm
    • Run on Kubernetes
    • Compose a scenario
    • Prepare and clean up with hooks
    • Prepare model weights
    • Prepare workload prerequisites
    • Inspect commands before execution
    • Read results and generate reports
  • Optimize configurations
    • Run a configuration search
    • Choose an objective and search space
    • Plan compute usage
    • Compare trade-offs and reuse configurations
  • Automate experiments
    • Run experiments with Python
    • Handle results and failures
    • Publish results and integrate with CI
  • Workloads
    • Training
      • MegatronBridge
      • Nemo Run
      • DDLB
    • Inference
      • vLLM
      • SGLang
      • AI Dynamo
      • SRT-Slurm
    • Networking and communication
      • NCCL
      • UCC
      • OSU
      • DeepEP Benchmark
      • NIXL Bench
      • NIXL EP
      • NIXL KVBench
      • NIXL CTPerf
    • Storage
      • fio
    • Simulation and prediction
      • AIConfigurator
      • DynamoMocker
      • Chakra Replay
    • General-purpose execution
      • Bash Command
      • Slurm Container
      • Sleep
    • Legacy integrations
      • NeMo Launcher (deprecated)
      • JaxToolbox workloads (DEPRECATED)
  • Extend CloudAI
    • Architecture and extension points
    • Add a workload
    • Add a reporter
    • Add an optimization agent
    • Add an execution system or runner
    • Develop and contribute
  • Reference
    • CLI reference
    • Workload and system support
    • Systems
      • NCCL on RunAI
    • System configuration example
    • Shared system settings
    • Test definitions
    • Scenario configuration
    • Agent and search settings
    • Reports
      • Training Report Schema
    • Python API
  • Troubleshooting
  • Workloads
  • Inference

Inference#

Measure serving behavior with your chosen inference engine and traffic pattern.

  • vLLM
  • SGLang
  • AI Dynamo
  • SRT-Slurm

previous

DDLB

next

vLLM
NVIDIA NVIDIA
Privacy Policy | Your Privacy Choices | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2026, NVIDIA CORPORATION & AFFILIATES.