OSU#

This workload (test_template_name is OSUBench) allows you to execute OSU Micro Benchmarks within the CloudAI framework.

Usage Example#

Test example:

name = "osu_example"
test_template_name = "OSUBench"
description = "OSU Benchmark example"

[cmd_args]
"docker_image_url" = "<docker container url here>"
"benchmarks_dir" = "/directory/with/osu/binaries/in/container"
"benchmark" = ["osu_allreduce", "osu_allgather"]
"iterations" = 10
"message_size" = "1024"

Test Scenario example:

name = "osu_example"

[[Tests]]
id = "Tests.1"
test_name = "osu_example"
num_nodes = "2"
time_limit = "00:20:00"

Test-in-Scenario example:

name = "osu-test"

[[Tests]]
id = "Tests.osu_allreduce"
num_nodes = 2
time_limit = "00:05:00"

name = "osu_example"
description = "OSU allreduce 1KB"
test_template_name = "OSUBench"

    [Tests.cmd_args]
    docker_image_url = "<docker container url here>"
    benchmarks_dir = "/directory/with/osu/binaries/in/container"
    benchmark = "osu_allreduce"
    iterations = 10
    message_size = "1024"

Comparison Reports#

The OSU comparison v2 report labels the X axis with the measured message sizes in bytes. Sizes are displayed at equally spaced positions, including zero-byte messages, rather than using default logarithmic ticks. When runs measure different size ranges, charts and tables use the sorted union of measured sizes. Missing measurements appear as chart gaps and n/a table cells.

API Documentation#

Command Arguments#

class cloudai.workloads.osu_bench.osu_bench.OSUBenchCmdArgs(

*,

docker_image_url: str,

benchmarks_dir: str,

benchmark: str | List[str],

message_size: str | List[str] | None = None,

iterations: int | None = None,

warmup: int | None = None,

mem_limit: int | None = None,

full: bool = True,

**extra_data: Any,

)[source]#

Bases: CmdArgs

Command line arguments for a OSU Benchmark test.

field docker_image_url: str [Required]#: URL of the Docker image to use for the test.

field benchmarks_dir: str [Required]#: Directory with the OSU Benchmark binaries inside the container.

field benchmark: str | List[str] [Required]#: Name of the benchmark to run.

field message_size: str | List[str] | None = None#

Message size for the benchmark.

Examples:

  // min = default, max = 128
128  // min = 2, max = 128
    // min 2, max = default

field iterations: int | None = None#: Number of iterations for the benchmark.

field warmup: int | None = None#: Number of warmup iterations to skip before timing.

field mem_limit: int | None = None#: Per-process maximum memory consumption in bytes.

field full: bool = True#: Print full format listing of results.

Test Definition#

class cloudai.workloads.osu_bench.osu_bench.OSUBenchTestDefinition(*, name: str, description: str, test_template_name: str, cmd_args: ~cloudai.workloads.osu_bench.osu_bench.OSUBenchCmdArgs, dse_excluded_args: list[str] = <factory>, extra_env_vars: dict[str, str | ~typing.List[str]] = {}, extra_cmd_args: dict[str, str] = {}, extra_container_mounts: list[str] = [], git_repos: list[~cloudai._core.installables.git_repo.GitRepo] = [], nsys: ~cloudai.models.workload.NsysConfiguration | None = None, predictor: ~cloudai.models.workload.PredictorConfig | None = None, training_report: ~cloudai.models.workload.TrainingReportConfig | None = None, agent: str = 'grid_search', agent_steps: int = 1, agent_metrics: list[str] = ['default'], agent_reward_function: str = 'inverse', agent_config: dict[str, ~typing.Any] | None = None, env_params: dict[str, ~cloudai.configurator.env_params.EnvParamSpec] = <factory>)[source]#

Bases: TestDefinition

Test definition for OSU Benchmark test.

property is_domain_randomization_enabled: bool#

at least one env_params annotation.

Type:: Whether the config declares domain randomization

is_dse_excluded_arg(path: str) → bool#: Return whether a dot-separated cmd_args path should be ignored by DSE.

is_env_sampled(cmd_args_path: str) → bool#: Whether a cmd_args field is env-sampled (env draws it per trial, not the agent).

metric_observations( system: System, tr: TestRun, ) → list[MetricObservation]#: Return canonical metric observations produced by this test run, if supported.

validator validate_env_params » all fields#

Validate env_params annotations against cmd_args.

env_params is an annotation: each key names a cmd_args field whose value is the candidate set (the single source of truth), and the entry carries only how to sample. So each key must name a real cmd_args field whose value is a candidate list; a scalar is already fixed, so annotating it is a meaningless label and is rejected here. When weights are declared, the list needs >= 2 values and the weights must align 1:1 with it. Sampling, persistence, the per-trial cmd_args overlay, and the cache key all live in CloudAIGymEnv; keeping this shape check in core lets the overlay stay agent- and workload-agnostic rather than re-implemented per workload.