Runtime Testing

Guides for generating, running, and reporting runtime tests.

• Overview and CLI
  • Quickstart
  • Why use runtime tests?
  • End-to-end usage (preferred)
  • CLI reference
  • runners_info.toml (maintained by user/CI)
  • Notes
• Authoring and Execution Deep Dive
  • Components
  • Test definitions (TOML)
  • Search functions
  • Run modes and runner selection
  • Writing a RuntimeTest
  • Recording results with TestExecutionResult
  • Execution flow
  • Tips
  • Pre-check-in workflow (recommended)
  • Additional examples
  • Troubleshooting (quick)
• Runner Configuration (runners_info.toml)
  • Purpose
  • File location
  • Required fields per runner
  • Recommended layout
  • Examples (Kit)
  • How batch_maker uses this file
  • Quick verification
  • Troubleshooting
• CI/CD Usage (multi-machine, direct CLI)
  • Goals
  • Stage 1: Generate jobs (CI machine)
  • Stage 2: Run jobs (target machine)
  • Stage 3: Generate reports (CI machine)
  • Notes and recommendations
  • Minimal YAML pseudo-template