AIConfigurator#

AIConfigurator workload (test_template_name is Aiconfigurator) runs the AIConfigurator predictor using the installed aiconfigurator Python package. It is a Standalone workload (no Slurm/Kubernetes/RunAI required).

Outputs#

Each test run produces:

  • report.json: Predictor output (JSON dict of metrics and metadata)

  • stdout.txt / stderr.txt: Predictor logs

  • run_simple_predictor.sh: Repro script containing the exact executed command (useful for debugging)

Usage Example#

Test TOML example (Disaggregated mode):

name = "aiconfigurator_disagg_demo"
description = "Example AIConfigurator disaggregated predictor"
test_template_name = "Aiconfigurator"

[cmd_args]
model_name = "LLAMA3.1_70B"
system = "h200_sxm"
backend = "trtllm"
version = "0.20.0"
isl = 4000
osl = 500

  [cmd_args.disagg]
  p_tp = 1
  p_pp = 1
  p_dp = 1
  p_bs = 1
  p_workers = 1

  d_tp = 1
  d_pp = 1
  d_dp = 1
  d_bs = 8
  d_workers = 2

  prefill_correction_scale = 1.0
  decode_correction_scale = 1.0

Test TOML example (Aggregated/IFB mode):

name = "aiconfigurator_agg_demo"
description = "Example AIConfigurator aggregated predictor"
test_template_name = "Aiconfigurator"

[cmd_args]
model_name = "LLAMA3.1_70B"
system = "h200_sxm"
backend = "trtllm"
version = "0.20.0"
isl = 4000
osl = 500

  [cmd_args.agg]
  batch_size = 8
  ctx_tokens = 16
  tp = 1
  pp = 1
  dp = 1

Running#

uv run cloudai run --system-config conf/common/system/standalone_system.toml \
   --tests-dir conf/experimental/aiconfigurator/test \
   --test-scenario conf/experimental/aiconfigurator/test_scenario/aiconfigurator_disagg.toml

API Documentation#

Command Arguments#

class cloudai.workloads.aiconfig.aiconfigurator.AiconfiguratorCmdArgs(
*,
model_name: str,
system: str,
backend: str = 'trtllm',
version: str = '0.20.0',
isl: int | List[int],
osl: int | List[int],
agg: Agg | None = None,
disagg: Disagg | None = None,
**extra_data: Any,
)[source]#

Bases: CmdArgs

Command arguments for Aiconfigurator workload with nested agg/disagg configs.

Test Definition#

class cloudai.workloads.aiconfig.aiconfigurator.AiconfiguratorTestDefinition(*, name: str, description: str, test_template_name: str, cmd_args: ~cloudai.workloads.aiconfig.aiconfigurator.AiconfiguratorCmdArgs, dse_excluded_args: list[str] = <factory>, extra_env_vars: dict[str, str | ~typing.List[str]] = {}, extra_cmd_args: dict[str, str] = {}, extra_container_mounts: list[str] = [], git_repos: list[~cloudai._core.installables.git_repo.GitRepo] = [], nsys: ~cloudai.models.workload.NsysConfiguration | None = None, predictor: ~cloudai.models.workload.PredictorConfig | None = None, agent: str = 'grid_search', agent_steps: int = 1, agent_metrics: list[str] = ['default'], agent_reward_function: str = 'inverse', agent_config: dict[str, ~typing.Any] | None = None)[source]#

Bases: TestDefinition

Test object for running Aiconfigurator predictor as a workload.

is_dse_excluded_arg(path: str) → bool#

Return whether a dot-separated cmd_args path should be ignored by DSE.

Command Generation Strategy (Standalone)#

class cloudai.workloads.aiconfig.standalone_command_gen_strategy.AiconfiguratorStandaloneCommandGenStrategy(
system: System,
test_run: TestRun,
)[source]#

Bases: CommandGenStrategy

Generate a standalone command that invokes the Aiconfigurator predictor and writes JSON output.

store_test_run() → None[source]#

Store the test run information in output folder.

Only at command generation time, CloudAI has all the information to store the test run.

gen_exec_command() → str[source]#

Generate the execution command for a test based on the given parameters.

Returns:

The generated execution command.

Return type:

str

__init__(
system: System,
test_run: TestRun,
) → None#
cleanup_job_artifacts() → None#

Best-effort cleanup hook run after the job has fully completed.

Report Generation Strategy#

class cloudai.workloads.aiconfig.report_generation_strategy.AiconfiguratorReportGenerationStrategy(
system: System,
tr: TestRun,
)[source]#

Bases: ReportGenerationStrategy

Generate metrics from Aiconfigurator predictor outputs.

__init__(
system: System,
tr: TestRun,
) → None#