Deployment#
Deployment covers how to run models as long-lived services that accept HTTP requests, batch concurrent callers, and integrate with application code or demos. This section is separate from Inference, which focuses on JSONL inference using AutoModel or Megatron-Bridge checkpoints for evaluation pipelines.
When to use which path#
Goal |
Playbook path |
Typical use |
|---|---|---|
Score eval JSONL, MCQ / LLM-judge |
Inference ( |
Reproducible benchmarks, multi-GPU batch runs |
OpenAI-compatible API, concurrent users, demos |
Deployment ( |
vLLM server, chat clients, load-balanced request queuing/handling |
Deployment recipes live in the Sports Intelligence Playbooks repo under avlm/deployment/.