benchmarks

Functions

run_aiperf_benchmark

Compatibility wrapper for callers that request one concurrency.

run_aiperf_sweep

Run a serving sweep while preserving offline and remote-code policy by default.

write_aiperf_report

ModeloptConfig BenchmarkResult

Bases: StrictModel

Show default config as JSON
Default config (JSON):

{
   "architecture_id": null,
   "checkpoint_dir": null,
   "solution_id": "unknown",
   "profile_id": "unknown",
   "topology_id": "unknown",
   "workload_id": "unknown",
   "gpu_count": 1,
   "cache_identity": "unknown",
   "engine": "aiperf",
   "topology": null,
   "workload": null,
   "concurrency": null,
   "metrics": null,
   "failures": 0,
   "raw_artifacts": null,
   "command": null,
   "started_at": null,
   "completed_at": null
}

field architecture_id: str [Required]
field cache_identity: str
field checkpoint_dir: str [Required]
field command: tuple[str, ...] [Required]
field completed_at: datetime [Optional]
field concurrency: int [Required]
Constraints:

  • ge = 1

field engine: Literal['aiperf']
field failures: int
Constraints:

  • ge = 0

field gpu_count: int
Constraints:

  • ge = 1

field metrics: dict[str, float] [Required]
field profile_id: str
field raw_artifacts: dict[str, str] [Required]
field solution_id: str
field started_at: datetime [Required]
field topology: dict[str, Any] [Required]
field topology_id: str
field workload: dict[str, Any] [Required]
field workload_id: str
run_aiperf_benchmark(checkpoint_dir, *, artifact_dir, concurrency, input_tokens, output_tokens, gpu_ids, topology, request_count, solution_id='unknown', profile_id='unknown', topology_id=None, **kwargs)

Compatibility wrapper for callers that request one concurrency.

Parameters:

  • checkpoint_dir (str | Path)

  • artifact_dir (str | Path)

  • concurrency (int)

  • input_tokens (int)

  • output_tokens (int)

  • gpu_ids (str)

  • topology (dict[str, Any])

  • request_count (int)

  • solution_id (str)

  • profile_id (str)

  • topology_id (str | None)

Return type:

BenchmarkResult

run_aiperf_sweep(checkpoint_dir, *, artifact_dir, concurrencies, input_tokens, output_tokens, gpu_ids, topology, request_counts=None, solution_id='unknown', profile_id='unknown', topology_id=None, executable='aiperf', endpoint_type='chat', extra_inputs=None, use_server_token_count=True, seed=42, readiness_timeout=1200, benchmark_timeout=600, gpu_telemetry='pynvml', trust_remote_code=False, allow_aiperf_v011_online_tokenizer_resolution=False)

Run a serving sweep while preserving offline and remote-code policy by default.

Parameters:

  • checkpoint_dir (str | Path)

  • artifact_dir (str | Path)

  • concurrencies (Iterable[int])

  • input_tokens (int)

  • output_tokens (int)

  • gpu_ids (str)

  • topology (dict[str, Any])

  • request_counts (dict[int, int] | None)

  • solution_id (str)

  • profile_id (str)

  • topology_id (str | None)

  • executable (str | Path)

  • endpoint_type (str)

  • extra_inputs (dict[str, Any] | None)

  • use_server_token_count (bool)

  • seed (int)

  • readiness_timeout (float)

  • benchmark_timeout (float)

  • gpu_telemetry (str | None)

  • trust_remote_code (bool)

  • allow_aiperf_v011_online_tokenizer_resolution (bool)

Return type:

list[BenchmarkResult]

write_aiperf_report(results, output_dir)
Parameters:

  • results (list[Any])

  • output_dir (str | Path)

Return type:

dict[str, str]