CloudAI#
The execution framework for AI-driven research.
Run the workloads your task requires, collect evidence, and use the results to guide the next experiment. Combine experiments into a broader investigation using existing workloads or your own.
Explore configurations with grid search, or connect your own optimization agent to guide search within your compute budget. Add workloads, systems, and reporters for your task.
Our internal optimization agents can significantly reduce the number of experiments needed to find strong configurations compared with grid search, saving time and compute budget. Contact the team for details.
Run your first experiment#
Start with an example below, or explore all supported workloads.
Benchmark Qwen3-0.6B on one GPU on a Slurm node.
uv run cloudai run --system-config slurm.toml \
--test-scenario conf/experimental/quickstart/vllm.toml
View configuration
name = "quickstart-vllm"
[[Tests]]
id = "vllm"
name = "vllm"
description = "Single-node Qwen inference benchmark"
test_template_name = "vllm"
num_nodes = 1
time_limit = "00:10:00"
[Tests.cmd_args]
docker_image_url = "nvcr.io#nvidia/ai-dynamo/vllm-runtime:1.1.1"
model = "Qwen/Qwen3-0.6B"
serve_healthcheck = "/health"
[Tests.cmd_args.decode]
gpu_ids = "0"
tensor_parallel_size = 1
[Tests.bench_cmd_args]
random_input_len = 16
random_output_len = 128
max_concurrency = 16
num_prompts = 30
Benchmark all-reduce on two Slurm nodes.
uv run cloudai run --system-config slurm.toml \
--test-scenario conf/experimental/quickstart/nccl.toml
View configuration
name = "quickstart-nccl"
[[Tests]]
id = "all-reduce"
name = "all-reduce"
description = "Two-node NCCL all-reduce benchmark"
test_template_name = "NcclTest"
num_nodes = 2
time_limit = "00:05:00"
[Tests.cmd_args]
docker_image_url = "nvcr.io#nvidia/pytorch:25.12-py3"
subtest_name = "all_reduce_perf_mpi"
ngpus = 1
minbytes = "8M"
maxbytes = "128M"
iters = 20
warmup_iters = 5
stepfactor = 2
Benchmark file I/O with Fio. Set the test file path to the storage you want to measure.
Standalone · Setup
uv run cloudai run --system-config standalone.toml \
--test-scenario conf/experimental/quickstart/fio.toml
Slurm · Setup
uv run cloudai run --system-config slurm.toml \
--test-scenario conf/experimental/quickstart/fio-slurm.toml
View configurations
Standalone
name = "quickstart-fio"
[[Tests]]
id = "fio"
name = "fio"
description = "Small file I/O benchmark"
test_template_name = "Fio"
num_nodes = 1
time_limit = "00:05:00"
[Tests.cmd_args]
fio_binary = "fio"
[Tests.cmd_args.args]
name = "cloudai-quickstart"
filename = "/tmp/cloudai-fio-quickstart"
rw = "randwrite"
bs = "128k"
size = "80m"
iodepth = 1
numjobs = 1
group_reporting = true
thread = true
Slurm
name = "quickstart-fio-slurm"
[[Tests]]
id = "fio"
name = "fio"
description = "Small file I/O benchmark"
test_template_name = "Fio"
num_nodes = 1
time_limit = "00:05:00"
extra_container_mounts = ["/tmp:/data"]
[Tests.cmd_args]
fio_binary = "fio"
docker_image_url = "openeuler/fio:3.42-oe2403sp3"
[Tests.cmd_args.args]
name = "cloudai-quickstart"
filename = "/data/cloudai-fio-quickstart"
rw = "randwrite"
bs = "128k"
size = "80m"
iodepth = 1
numjobs = 1
group_reporting = true
thread = true
Run ten Qwen3-30B-A3B training steps on two eight-GPU H100 Slurm nodes.
Set HF_TOKEN in your environment.
uv run cloudai run --system-config slurm.toml \
--test-scenario conf/experimental/quickstart/mbridge.toml
View configuration
name = "quickstart-mbridge"
[[Tests]]
id = "training"
name = "training"
description = "Two-node H100 Qwen training benchmark"
test_template_name = "MegatronBridge"
num_nodes = 2
time_limit = "00:30:00"
[[Tests.git_repos]]
url = "https://github.com/NVIDIA-NeMo/Megatron-Bridge.git"
commit = "r0.4.0"
mount_as = "/opt/Megatron-Bridge"
init_submodules = true
[Tests.cmd_args]
gpu_type = "h100"
gpus_per_node = 8
num_gpus = 16
container_image = "nvcr.io#nvidia/nemo:26.02.00"
model_family_name = "qwen"
model_recipe_name = "qwen3_30b_a3b"
domain = "llm"
task = "pretrain"
compute_dtype = "fp8_cs"
max_steps = 10
nemorun_version = "v0.9.0"
[Tests.extra_env_vars]
PYTHONPATH = "/opt/Megatron-Bridge/3rdparty/Megatron-LM:${PYTHONPATH}"
Start here#
Install uv 0.13.4 or newer and clone CloudAI:
git clone https://github.com/NVIDIA/cloudai.git
cd cloudai
Standalone#
Run Fio locally with fio installed.
cp conf/common/system/standalone_system.toml standalone.toml
On Ubuntu, install Fio with sudo apt-get install fio.
Slurm#
On your Slurm submit host, prepare the container prerequisites and copy the system configuration:
cp conf/common/system/example_slurm_cluster.toml slurm.toml
Set your partitions, nodes, and GPU counts in slurm.toml.
For Fio, set the host path in extra_container_mounts to the storage you want to measure.
See Systems for system settings and Install CloudAI for installation alternatives.
Read your results#
Open the results directory printed by CloudAI. See Read results and generate reports for HTML reports and experiment.json.
Use the Python API to automate experiments and consume results.
Make CloudAI fit your task: build an extension or contribute an improvement.
Notice and Disclaimer
NOTICE AND DISCLAIMER: This software automatically retrieves, accesses or interacts with external materials. Those retrieved materials are not distributed with this software and are governed solely by separate terms, conditions and licenses. You are solely responsible for finding, reviewing and complying with all applicable terms, conditions, and licenses, and for verifying the security, integrity and suitability of any retrieved materials for your specific use case. This software is provided “AS IS”, without warranty of any kind. The author makes no representations or warranties regarding any retrieved materials, and assumes no liability for any losses, damages, liabilities or legal consequences from your use or inability to use this software or any retrieved materials. Use this software and the retrieved materials at your own risk.