xr_ai_model_server_stack#

Public convenience API for sample-local inference stacks.

Functions#

run_model_stack(→ None)

Start or stop the configured local inference stack for a developer sample.

Package Contents#

xr_ai_model_server_stack.run_model_stack( base: pathlib.Path, services: tuple[str, Ellipsis], *, stop: bool = False, gpu_profile: str | None = None, ) → None#

Start or stop the configured local inference stack for a developer sample.

base is the sample’s model-server project directory, conventionally <repository>/agent-samples/<sample>/model-servers or <repository>/apps/<application>/model-servers. Relative paths are resolved against the current working directory. Service projects are resolved under that repository’s services/ directory.

services selects from speaker-stt, stt, tts, omni, vlm, and embedding. The selected base/yaml/<gpu_profile>.yaml must contain exactly those server mappings. Each mapping overrides the corresponding shared server YAML. Launch order is speaker ASR, batch STT, TTS, Omni, VLM, then embedding, omitting unselected services.

gpu_profile selects a checked-in profile by filename stem; None detects the hardware automatically for startup. stop=False requires HF_TOKEN, waits for every server to become ready, and returns with servers running. stop=True needs neither GPU detection nor download credentials. With no explicit profile it uses the selected services’ shared default ports; pass gpu_profile to honor profile-specific port overrides. It stops only recognized servers on those selected ports. It can stop servers used by another sample sharing those ports; no launch history is recorded.

Stop the current stack before switching samples, editing its server settings, or updating the checkout. This convenience API does not read consumer endpoint profiles or launch independently deployed NIM endpoints.

Raises:: ValueError: The selected profile or server configuration is invalid. RuntimeError: GPU inventory or configured server cleanup fails. SystemExit: Required credentials are absent or stack startup aborts.