xr_ai_models#

Unified typed service protocols and model clients for XR AI.

Repository code talks to the typed *Service protocols. The concrete OpenAICompat* clients cover chat, embeddings, and HTTP speech; OCR uses the Nemotron v2 protocol and Riva speech uses optional gRPC.

Submodules#

xr_ai_models.presets

Attributes#

`ContentPart`	A supported text, image, or video part in a chat message.
`ImageInput`	Image bytes, a filesystem path, a `data:` URL, or an HTTP(S) URL.
`VideoInput`	Video bytes, a filesystem path, a `data:` URL, or an HTTP(S) URL.
`Category`	A model role supported by `ModelsConfig`.
`KIND_OPENAI_COMPAT`	The adapter kind for OpenAI-compatible HTTP endpoints.
`KIND_SYSTEMONE`	The adapter kind for typed Clef SystemOne decision endpoints.
`ModelKind`	A supported model-service adapter implementation.
`Spec`	Any typed model-role specification stored in `ModelsConfig`.
`ThinkingControl`	How a model receives the reasoning switch.

Exceptions#

`DecisionInputTooLargeError`	The decision service rejected a request as more input than its model accepts.
`ModelServiceError`	Base for model-service failures, including embedding identity and OCR errors.
`OCRRequestError`	OCR input could not be read or the HTTP request failed.
`OCRResponseError`	The OCR endpoint returned malformed JSON or invalid detection data.

Classes#

`Capabilities`	Features supported by a configured model endpoint.
`ChatMessage`	One input turn in an LLM or VLM conversation.
`ChatResponse`	A normalized non-streaming response from a chat model.
`DecisionAnswer`	One validated choice and its probability trace.
`DecisionQuestion`	One choice question for a structured decision model.
`DecisionResponse`	Validated answers and metadata returned by a decision service.
`DecisionService`	Structural interface for typed, choice-only decision services.
`ChatStreamEvent`	One normalized event from a streamed chat completion.
`EmbeddingService`	Structural interface for text-embedding services.
`ImagePart`	An image reference in a multimodal chat message.
`LLMService`	Structural interface for text chat-completion services.
`OCRService`	Single-image OCR shared by local Hugging Face and NIM v2 servers.
`OCRRegion`	One recognized text region; polygon coordinates are normalized to [0, 1].
`STTService`	Structural interface for speech-to-text services.
`TextPart`	Text content in a multimodal chat message.
`ToolCall`	A function-tool invocation requested by a model.
`ToolDef`	A function tool definition exposed to a chat model.
`TTSService`	Structural interface for text-to-speech services.
`VideoPart`	A video reference in a multimodal chat message.
`VLMService`	Structural interface for visual chat-completion services.
`OpenAICompatLLM`	OpenAI-compatible `/v1/chat/completions` client.
`OpenAICompatEmbedding`	OpenAI-compatible `/v1/embeddings` client.
`OpenAICompatSTT`	OpenAI-compatible `/v1/audio/transcriptions` client.
`OpenAICompatTTS`	OpenAI-compatible `/v1/audio/speech` client.
`OpenAICompatVLM`	OpenAI-compatible client for image- and video-bearing chat requests.
`AdapterSpec`	API dialect and model-specific request/response behavior.
`DecisionSpec`	Configuration for a choice-only structured decision endpoint.
`DeploymentSpec`	Process ownership for the endpoint that serves a model role.
`EmbeddingSpec`	Configuration for a text-embedding model role.
`EndpointSpec`	Connectivity, authentication, timeout, and readiness for an adapter.
`LLMSpec`	Configuration for a text chat-completion model role.
`ModelsConfig`	Logical-name to typed model-role specifications.
`OCRSpec`	Configuration for OCR via the Nemotron v2 HTTP protocol.
`STTSpec`	Configuration for a speech-to-text model role.
`TTSSpec`	Configuration for a text-to-speech model role.
`VLMSpec`	Configuration for an image- or video-language model role.
`ThinkingSpec`	How a model endpoint switches reasoning, and its default state.
`RivaSTT`	Riva ASR client: offline (batch) recognition over gRPC.
`RivaTTS`	Riva TTS client with buffered WAV or PCM synthesis.

Functions#

`load_models_config`(→ ModelsConfig)	Load a YAML or JSON model profile and resolve its adapter presets.
`load_models_config_from_dict`(→ ModelsConfig)	Build a model configuration from a parsed direct or `models` mapping.
`make_decision`(→ xr_ai_models.DecisionService)	Construct the typed decision service for the named configuration entry.
`make_embedding`(→ xr_ai_models.EmbeddingService)	Construct the embedding service for the named configuration entry.
`make_llm`(→ xr_ai_models.LLMService)	Construct the text chat service for the named configuration entry.
`make_ocr`(→ xr_ai_models.OCRService)	Construct OCR for name; reject a missing role, wrong role, or unsupported kind.
`make_stt`(→ xr_ai_models.STTService)	Construct the speech-to-text service for the named configuration entry.
`make_tts`(→ xr_ai_models.TTSService)	Construct the text-to-speech service for the named configuration entry.
`make_vlm`(→ xr_ai_models.VLMService)	Construct the visual chat service for the named configuration entry.

Package Contents#

class xr_ai_models.Capabilities#

Features supported by a configured model endpoint.

streaming: bool = True#: Whether the endpoint supports token streaming.

tool_calls: bool = False#: Whether the endpoint supports function-tool calls.

vision: bool = False#: Whether the endpoint accepts image inputs.

video: bool = False#: Whether the endpoint accepts video inputs.

reasoning: bool = False#: Whether the endpoint can return model reasoning.

class xr_ai_models.ChatMessage#

One input turn in an LLM or VLM conversation.

role: Literal['system', 'user', 'assistant', 'tool']#: The participant role associated with the message.

content: str | list[ContentPart]#: Plain text or ordered multimodal content supplied by the participant.

tool_calls: list[ToolCall] | None = None#: Tool calls emitted by an assistant turn, when present.

tool_call_id: str | None = None#: The call identifier answered by a tool-result message, when applicable.

class xr_ai_models.ChatResponse#

A normalized non-streaming response from a chat model.

content: str#: The assistant’s user-visible response text.

reasoning: str | None#: Normalized model reasoning, when the endpoint returns it.

tool_calls: list[ToolCall] | None#: Function-tool invocations requested by the model, when present.

finish_reason: str | None#: The provider’s reason for ending generation, when supplied.

raw: dict[str, Any]#: The unmodified provider response object.

class xr_ai_models.DecisionAnswer#

One validated choice and its probability trace.

choice: str#: The selected label from the question’s criteria.

confidence: float#: The selected choice probability.

probabilities: Mapping[str, float]#: Probability for every permitted choice label.

exception xr_ai_models.DecisionInputTooLargeError#

The decision service rejected a request as more input than its model accepts.

The service owns that limit, and the caller cannot know it in advance. The same request will not succeed; a caller can retry with less input.

class xr_ai_models.DecisionQuestion#

One choice question for a structured decision model.

The keys in criteria are the permitted answer labels. Each value gives the criterion that distinguishes that choice. Define at least two non-empty labels with non-empty descriptions.

instructions: str#: Question-specific instructions for selecting a criterion.

criteria: Mapping[str, str]#: Permitted choice labels and their selection criteria.

class xr_ai_models.DecisionResponse#

Validated answers and metadata returned by a decision service.

answers: Mapping[str, DecisionAnswer]#: Answers keyed by the question identifiers supplied to decide.

model: str#: Model identifier reported by the service.

usage: Mapping[str, Any]#: Provider-reported usage metadata.

raw: Mapping[str, Any]#: Unmodified JSON response from the provider.

class xr_ai_models.DecisionService#

Structural interface for typed, choice-only decision services.

async decide( state: str | Mapping[str, Any], questions: Mapping[str, DecisionQuestion], *, images: Sequence[ImageInput] = (), timeout: float | None = None, ) → DecisionResponse#

Evaluate all questions against text or a JSON-compatible state mapping.

Optional images are ordered JPEG/PNG bytes, local pathlib.Path objects, or inline base64 data URL strings. Files must match their JPEG or PNG suffix. Filesystem path strings and remote URLs are rejected; the server never fetches images. Describe image roles and ordering in state or instructions. Empty images preserve the text-only wire contract. Older text-only endpoints reject image requests; there is no silent fallback.

Return one choice and a complete probability map for each question. No free-form text or reasoning is generated. A per-call timeout must be positive and finite; the SystemOne adapter caps it at the configured endpoint timeout. Transport failures and invalid or incomplete answers raise exceptions without returning partial decisions.

Raises:

DecisionInputTooLargeError: The service rejected the request as more: input than its model accepts; a smaller request may succeed.

async health() → bool#: Return whether the configured endpoint is ready for requests.

async close() → None#: Release resources owned by the service.

class xr_ai_models.ChatStreamEvent#

One normalized event from a streamed chat completion.

Text and reasoning events carry deltas as they arrive. Tool calls are emitted only after the provider finishes the response and the complete batch has been validated. A finish event carries the provider finish reason.

kind: Literal['text', 'reasoning', 'tool_call', 'finish']#: The kind of streamed event.

text: str | None = None#: A user-visible text delta for kind == "text".

reasoning: str | None = None#: A model reasoning delta for kind == "reasoning".

tool_call: ToolCall | None = None#: A complete tool call for kind == "tool_call".

finish_reason: str | None = None#: The provider finish reason for kind == "finish".

xr_ai_models.ContentPart#: A supported text, image, or video part in a chat message.

class xr_ai_models.EmbeddingService#

Structural interface for text-embedding services.

async model_identity() → str | None#

Return a stable backing-model identity, or None when absent.

None means a successful lookup found no identity; it must not hide a failed operation. This is not a weight checksum: deployments that replace weights under the same identity need an explicit cache revision. Consumers choose whether absence permits operation without caching.

Raises:

ModelServiceError: Identity lookup failed or returned invalid: metadata. Consumers own any fallback or retry policy.

async embed( texts: Sequence[str], *, timeout: float | None = None, ) → list[list[float]]#: Embed text strings, returning one vector for each input in order.

async health() → bool#: Return whether the configured endpoint is ready for requests.

async close() → None#: Release resources owned by the service.

exception xr_ai_models.ModelServiceError#: Base for model-service failures, including embedding identity and OCR errors.

xr_ai_models.ImageInput#: Image bytes, a filesystem path, a data: URL, or an HTTP(S) URL.

class xr_ai_models.ImagePart#

An image reference in a multimodal chat message.

url: str#: An HTTP(S) or data: URL containing the image.

type: Literal['image_url'] = 'image_url'#: The OpenAI-compatible discriminator for this content part.

class xr_ai_models.LLMService#

Structural interface for text chat-completion services.

enable_thinking requests or suppresses model reasoning for one call; None uses the endpoint’s configured default. The endpoint renders it in the model’s own dialect, and models without a switch ignore it. thinking_budget caps reasoning tokens on servers that enforce it.

capabilities: Capabilities#: Features supported by this service.

async chat( messages: Sequence[ChatMessage], *, tools: Sequence[ToolDef] | None = None, max_tokens: int | None = None, temperature: float | None = None, enable_thinking: bool | None = None, thinking_budget: int | None = None, timeout: float | None = None, headers: Mapping[str, str] | None = None, ) → ChatResponse#: Generate one complete response for a sequence of chat messages.

stream( messages: Sequence[ChatMessage], *, tools: Sequence[ToolDef] | None = None, max_tokens: int | None = None, temperature: float | None = None, enable_thinking: bool | None = None, thinking_budget: int | None = None, timeout: float | None = None, headers: Mapping[str, str] | None = None, ) → AsyncIterator[str]#: Stream user-visible response text for a sequence of chat messages.

stream_events( messages: Sequence[ChatMessage], *, tools: Sequence[ToolDef] | None = None, max_tokens: int | None = None, temperature: float | None = None, enable_thinking: bool | None = None, thinking_budget: int | None = None, timeout: float | None = None, headers: Mapping[str, str] | None = None, ) → AsyncIterator[ChatStreamEvent]#

Stream text, reasoning, complete tool calls, and finish events.

Tool calls are held until the stream completes successfully, so a truncated or malformed function call is never exposed as executable.

async health() → bool#: Return whether the configured endpoint is ready for requests.

async close() → None#: Release resources owned by the service.

class xr_ai_models.OCRService#

Single-image OCR shared by local Hugging Face and NIM v2 servers.

async read_text( image: ImageInput, *, merge_level: Literal['word', 'sentence', 'paragraph'] = 'word', timeout: float | None = None, ) → list[OCRRegion]#

Transcribe inline PNG/JPEG bytes, a Path, or a base64 data URL.

Remote URLs are not accepted. Regions follow backend reading order. This method does not crop, sharpen, parse fields, or change resolution.

Raises:

OCRRequestError: Input-file, transport, timeout, or HTTP failure.

OCRResponseError: The endpoint returned malformed OCR data.

async health() → bool#: Return whether the configured endpoint is ready.

async close() → None#: Release owned client resources.

class xr_ai_models.OCRRegion#

One recognized text region; polygon coordinates are normalized to [0, 1].

text: str#: Transcribed text.

confidence: float#: Recognition confidence reported by the backend.

polygon: tuple[tuple[float, float], Ellipsis]#: Ordered (x, y) vertices in the input image’s coordinate system.

class xr_ai_models.STTService#

Structural interface for speech-to-text services.

async transcribe( audio: bytes, *, sample_rate: int | None = None, channels: int = 1, timeout: float | None = None, ) → str#: Transcribe WAV data or 16-bit PCM audio into text.

async health() → bool#: Return whether the configured endpoint is ready for requests.

async close() → None#: Release resources owned by the service.

class xr_ai_models.TextPart#

Text content in a multimodal chat message.

text: str#: The text presented to the model.

type: Literal['text'] = 'text'#: The OpenAI-compatible discriminator for this content part.

class xr_ai_models.ToolCall#

A function-tool invocation requested by a model.

id: str#: The provider-assigned identifier used to submit the tool result.

name: str#: The function name requested by the model.

arguments: str#: JSON-encoded arguments string, per the OpenAI tool-call contract.

class xr_ai_models.ToolDef#

A function tool definition exposed to a chat model.

name: str#: The function name the model uses in a ToolCall.

description: str#: A natural-language description of what the function does.

parameters: dict[str, Any]#: The function parameters as a JSON Schema object.

to_openai() → dict[str, Any]#: Return this definition in OpenAI’s function-tool wire format.

class xr_ai_models.TTSService#

Structural interface for text-to-speech services.

async synthesize( text: str, *, response_format: str = 'wav', timeout: float | None = None, ) → bytes#: Synthesize text and return audio in the requested format.

async health() → bool#: Return whether the configured endpoint is ready for requests.

async close() → None#: Release resources owned by the service.

xr_ai_models.VideoInput#: Video bytes, a filesystem path, a data: URL, or an HTTP(S) URL.

class xr_ai_models.VideoPart#

A video reference in a multimodal chat message.

url: str#: An HTTP(S) or data: URL containing the video.

type: Literal['video_url'] = 'video_url'#: The OpenAI-compatible discriminator for this content part.

class xr_ai_models.VLMService#

Structural interface for visual chat-completion services.

capabilities: Capabilities#: Features supported by this service.

async ask_image( image: ImageInput, question: str, *, system_prompt: str = '', max_tokens: int | None = None, temperature: float | None = None, timeout: float | None = None, headers: Mapping[str, str] | None = None, ) → ChatResponse#: Generate one response to a question about an image.

async ask_images( images: Sequence[ImageInput], question: str, *, system_prompt: str = '', max_tokens: int | None = None, temperature: float | None = None, timeout: float | None = None, headers: Mapping[str, str] | None = None, ) → ChatResponse#: Generate one response to a question about multiple images.

async ask_video( video: VideoInput, question: str, *, system_prompt: str = '', max_tokens: int | None = None, temperature: float | None = None, timeout: float | None = None, headers: Mapping[str, str] | None = None, ) → ChatResponse#: Generate one response to a question about a video.

stream( image: ImageInput, question: str, *, system_prompt: str = '', max_tokens: int | None = None, temperature: float | None = None, timeout: float | None = None, headers: Mapping[str, str] | None = None, ) → AsyncIterator[str]#: Stream response text for a question about an image.

stream_images( images: Sequence[ImageInput], question: str, *, system_prompt: str = '', max_tokens: int | None = None, temperature: float | None = None, timeout: float | None = None, headers: Mapping[str, str] | None = None, ) → AsyncIterator[str]#: Stream response text for a question about multiple images.

async health() → bool#: Return whether the configured endpoint is ready for requests.

async close() → None#: Release resources owned by the service.

class xr_ai_models.OpenAICompatLLM( base_url: str, model_name: str, *, capabilities: xr_ai_models.Capabilities | None = None, reasoning_field: str | None = None, default_extras: dict[str, Any] | None = None, thinking: xr_ai_models.ThinkingSpec | None = None, api_key_env: str | None = None, timeout: float = 60.0, health_check: bool = True, health_path: str = '/health', client: httpx.AsyncClient | None = None, )#

OpenAI-compatible /v1/chat/completions client.

Used directly for plain LLMs (Nemotron3-Nano, Nemotron-Omni) and indirectly via OpenAICompatVLM for VLMs. The API key is read once from api_key_env during construction. An injected HTTP client remains owned by the caller and is not closed here.

health_url#: The URL used to check endpoint readiness.

capabilities#: Features declared for this model endpoint.

async chat( messages: Sequence[xr_ai_models.ChatMessage], *, tools: Sequence[xr_ai_models.ToolDef] | None = None, max_tokens: int | None = None, temperature: float | None = None, enable_thinking: bool | None = None, thinking_budget: int | None = None, timeout: float | None = None, headers: Mapping[str, str] | None = None, ) → xr_ai_models.ChatResponse#

Generate and return one normalized chat response.

Per-call generation values override endpoint defaults. headers may supply request context but cannot override the configured authorization header.

async stream( messages: Sequence[xr_ai_models.ChatMessage], *, tools: Sequence[xr_ai_models.ToolDef] | None = None, max_tokens: int | None = None, temperature: float | None = None, enable_thinking: bool | None = None, thinking_budget: int | None = None, timeout: float | None = None, headers: Mapping[str, str] | None = None, ) → AsyncIterator[str]#

Stream user-visible text deltas from a chat completion.

Malformed server-sent event lines and deltas without content are skipped. Tool-call and reasoning deltas are not yielded.

async stream_events( messages: Sequence[xr_ai_models.ChatMessage], *, tools: Sequence[xr_ai_models.ToolDef] | None = None, max_tokens: int | None = None, temperature: float | None = None, enable_thinking: bool | None = None, thinking_budget: int | None = None, timeout: float | None = None, headers: Mapping[str, str] | None = None, ) → AsyncIterator[xr_ai_models.ChatStreamEvent]#

Stream normalized events, withholding tool calls until completion.

A provider finish reason followed by [DONE] is required before any tool call is emitted. This prevents consumers from executing partial calls if the connection ends early or the provider returns invalid tool data.

async health() → bool#

Return whether the endpoint health check succeeds.

Returns True without an HTTP request when health checks were disabled at construction time.

async close() → None#: Close the internally created HTTP client, if one is owned.

class xr_ai_models.OpenAICompatEmbedding( base_url: str, model_name: str, *, api_key_env: str | None = None, timeout: float = 60.0, health_check: bool = True, health_path: str = '/health', client: httpx.AsyncClient | None = None, )#

OpenAI-compatible /v1/embeddings client.

The API key is read once from api_key_env during construction. An injected HTTP client remains owned by the caller and is not closed here.

async model_identity() → str | None#

Read the configured model’s backing root from /v1/models.

Return None only when a successful response has no matching model or no backing identity. A served alias alone does not identify weights. Uses the HTTP client’s configured timeout; callers own fallback policy.

Raises:

ModelServiceError: The request failed or the response contains: invalid or ambiguous metadata. The original transport or JSON exception is retained as the cause when available.

async embed( texts: Sequence[str], *, timeout: float | None = None, ) → list[list[float]]#

Embed texts and return one vector per input in input order.

An empty input is handled locally. A response containing a different number of vectors raises ValueError.

async health() → bool#

Return whether the endpoint health check succeeds.

Returns True without an HTTP request when health checks were disabled at construction time.

async close() → None#: Close the internally created HTTP client, if one is owned.

class xr_ai_models.OpenAICompatSTT( base_url: str, *, api_key_env: str | None = None, timeout: float = 30.0, health_check: bool = True, health_path: str = '/health', client: httpx.AsyncClient | None = None, )#

OpenAI-compatible /v1/audio/transcriptions client.

The API key is read once from api_key_env during construction. An injected HTTP client remains owned by the caller and is not closed here.

async transcribe( audio: bytes, *, sample_rate: int | None = None, channels: int = 1, timeout: float | None = None, ) → str#

Transcribe WAV data or 16-bit PCM audio into text.

When sample_rate is provided, audio is interpreted as raw signed 16-bit PCM with the given channel count and wrapped in a WAV container. Otherwise, audio must already contain a server-supported audio file.

async health() → bool#

Return whether the endpoint health check succeeds.

Returns True without an HTTP request when health checks were disabled at construction time.

async close() → None#: Close the internally created HTTP client, if one is owned.

class xr_ai_models.OpenAICompatTTS( base_url: str, *, api_key_env: str | None = None, timeout: float = 30.0, health_check: bool = True, health_path: str = '/health', client: httpx.AsyncClient | None = None, )#

OpenAI-compatible /v1/audio/speech client.

The API key is read once from api_key_env during construction. An injected HTTP client remains owned by the caller and is not closed here.

async synthesize( text: str, *, response_format: str = 'wav', timeout: float | None = None, ) → bytes#: Synthesize text and return audio bytes in response_format.

async health() → bool#

Return whether the endpoint health check succeeds.

Returns True without an HTTP request when health checks were disabled at construction time.

async close() → None#: Close the internally created HTTP client, if one is owned.

class xr_ai_models.OpenAICompatVLM( base_url: str, model_name: str, *, capabilities: xr_ai_models.Capabilities | None = None, default_extras: dict[str, Any] | None = None, thinking: xr_ai_models.ThinkingSpec | None = None, api_key_env: str | None = None, timeout: float = 60.0, health_check: bool = True, health_path: str = '/health', client: httpx.AsyncClient | None = None, )#

OpenAI-compatible client for image- and video-bearing chat requests.

Image and video paths or bytes are converted to data: URLs before submission. An injected HTTP client remains owned by the caller and is not closed here.

property capabilities: xr_ai_models.Capabilities#: Return the features declared for the underlying chat endpoint.

property health_url: str#: Return the URL used to check endpoint readiness.

async ask_image( image: xr_ai_models.ImageInput, question: str, *, system_prompt: str = '', max_tokens: int | None = None, temperature: float | None = None, timeout: float | None = None, headers: Mapping[str, str] | None = None, ) → xr_ai_models.ChatResponse#: Generate one response to question about image.

async ask_images( images: Sequence[xr_ai_models.ImageInput], question: str, *, system_prompt: str = '', max_tokens: int | None = None, temperature: float | None = None, timeout: float | None = None, headers: Mapping[str, str] | None = None, ) → xr_ai_models.ChatResponse#

Generate one response to question about an ordered image sequence.

Raises ValueError when images is empty.

async ask_video( video: xr_ai_models.VideoInput, question: str, *, system_prompt: str = '', max_tokens: int | None = None, temperature: float | None = None, timeout: float | None = None, headers: Mapping[str, str] | None = None, ) → xr_ai_models.ChatResponse#

Generate one response to question about video.

Raises ValueError unless video support was declared in the configured capabilities.

async stream( image: xr_ai_models.ImageInput, question: str, *, system_prompt: str = '', max_tokens: int | None = None, temperature: float | None = None, timeout: float | None = None, headers: Mapping[str, str] | None = None, ) → AsyncIterator[str]#: Stream response text for question about image.

async stream_images( images: Sequence[xr_ai_models.ImageInput], question: str, *, system_prompt: str = '', max_tokens: int | None = None, temperature: float | None = None, timeout: float | None = None, headers: Mapping[str, str] | None = None, ) → AsyncIterator[str]#

Stream response text for a question about an ordered image sequence.

Raises ValueError when images is empty.

async health() → bool#: Return whether the underlying chat endpoint is ready.

async close() → None#: Release resources owned by the underlying chat client.

class xr_ai_models.AdapterSpec#

API dialect and model-specific request/response behavior.

The riva_grpc speech fields ride here: function_id (hosted NVCF selects the model by function-id metadata), use_ssl, language, and the TTS-only voice and sample_rate.

kind: ModelKind = 'openai_compat'#: The concrete client implementation used for this role.

model_name: str = ''#: The model identifier sent to endpoints that require one.

reasoning_field: str | None = None#: The response field containing reasoning, or None for auto-detection.

capabilities: dict[str, Any]#: Overrides used to construct the service’s capability flags.

default_extras: dict[str, Any]#: Model-specific fields merged into every request payload.

function_id: str | None = None#: NVCF function id for hosted Riva speech endpoints.

use_ssl: bool = False#: Whether the Riva gRPC channel uses TLS.

language: str = 'en-US'#: BCP-47 language code for Riva speech recognition and synthesis.

voice: str = ''#: Riva TTS voice name; the service default when empty.

sample_rate: int = 44100#: Riva TTS output sample rate in Hz.

thinking: xr_ai_models.ThinkingSpec#: How the model switches reasoning, and its default state.

xr_ai_models.Category#: A model role supported by ModelsConfig.

class xr_ai_models.DecisionSpec#

Configuration for a choice-only structured decision endpoint.

adapter: AdapterSpec#: Model identifier and adapter settings.

endpoint: EndpointSpec#: Endpoint connectivity, authentication, and readiness settings.

deployment: DeploymentSpec#: Launcher ownership metadata for the serving process.

class xr_ai_models.DeploymentSpec#

Process ownership for the endpoint that serves a model role.

Client profiles can omit deployment: the default external ownership leaves the endpoint’s process lifecycle to its operator. Shared model-server profiles use managed to select processes to launch. Explicit reused metadata remains supported for older profiles.

credentials names keys the launched service itself needs (e.g. NGC_API_KEY for a NIM container’s nvcr.io pull and engine download) even when the endpoint takes no API key.

ownership: Ownership = 'external'#: Whether the launcher starts, reuses, or does not manage the service.

service: str | None = None#: The launcher service name for managed or reused deployments.

credentials: tuple[str, Ellipsis] = ()#: Environment keys the launched service needs (collected by the launcher).

class xr_ai_models.EmbeddingSpec( kind: ModelKind = KIND_OPENAI_COMPAT, base_url: str = '', model_name: str = '', api_key_env: str | None = None, timeout: float = 60.0, health_check: bool = True, deployment: DeploymentSpec | None = None, *, adapter: AdapterSpec | None = None, endpoint: EndpointSpec | None = None, )#

Configuration for a text-embedding model role.

adapter: AdapterSpec#: Model-specific request and response behavior.

endpoint: EndpointSpec#: Endpoint connectivity, authentication, and readiness settings.

deployment: DeploymentSpec#: Launcher ownership metadata for the serving process.

class xr_ai_models.EndpointSpec#

Connectivity, authentication, timeout, and readiness for an adapter.

base_url: str = ''#: The endpoint root URL, without a model-specific route.

api_key_env: str | None = None#: Environment variable containing the bearer token, when required.

timeout: float = 60.0#: Default request timeout in seconds.

readiness: Readiness = 'health'#: Policy for explicit health() calls; does not schedule startup probes.

health_path: str = '/health'#: Path used by explicit HTTP health() calls (NIM: /v1/health/ready).

property health_check: bool#: Whether an explicit health() call probes the endpoint.

xr_ai_models.KIND_OPENAI_COMPAT: ModelKind = 'openai_compat'#: The adapter kind for OpenAI-compatible HTTP endpoints.

xr_ai_models.KIND_SYSTEMONE: ModelKind = 'systemone'#: The adapter kind for typed Clef SystemOne decision endpoints.

class xr_ai_models.LLMSpec( kind: ModelKind = KIND_OPENAI_COMPAT, base_url: str = '', model_name: str = '', api_key_env: str | None = None, reasoning_field: str | None = None, capabilities: dict[str, Any] | None = None, default_extras: dict[str, Any] | None = None, timeout: float = 60.0, health_check: bool = True, deployment: DeploymentSpec | None = None, *, adapter: AdapterSpec | None = None, endpoint: EndpointSpec | None = None, thinking: xr_ai_models.ThinkingSpec | None = None, )#

Configuration for a text chat-completion model role.

adapter: AdapterSpec#: Model-specific request and response behavior.

endpoint: EndpointSpec#: Endpoint connectivity, authentication, and readiness settings.

deployment: DeploymentSpec#: Launcher ownership metadata for the serving process.

xr_ai_models.ModelKind#: A supported model-service adapter implementation.

class xr_ai_models.ModelsConfig#

Logical-name to typed model-role specifications.

entries: dict[str, Spec]#: Model-role specifications keyed by application-defined logical name.

llm(name: str) → LLMSpec#: Return the LLM specification named name.

vlm(name: str) → VLMSpec#: Return the VLM specification named name.

stt(name: str) → STTSpec#: Return the speech-to-text specification named name.

tts(name: str) → TTSSpec#: Return the text-to-speech specification named name.

embedding(name: str) → EmbeddingSpec#: Return the embedding specification named name.

ocr(name: str) → OCRSpec#: Return the OCR specification named name.

decision(name: str) → DecisionSpec#: Return the decision specification named name.

property required_credentials: tuple[str, Ellipsis]#: Return the sorted environment-variable names required by all roles.

class xr_ai_models.OCRSpec#

Configuration for OCR via the Nemotron v2 HTTP protocol.

adapter: AdapterSpec#: OCR request dialect.

endpoint: EndpointSpec#: Endpoint connectivity and readiness.

deployment: DeploymentSpec#: Launcher ownership metadata.

xr_ai_models.Spec#: Any typed model-role specification stored in ModelsConfig.

class xr_ai_models.STTSpec( kind: ModelKind = KIND_OPENAI_COMPAT, base_url: str = '', api_key_env: str | None = None, timeout: float = 30.0, health_check: bool = True, deployment: DeploymentSpec | None = None, *, adapter: AdapterSpec | None = None, endpoint: EndpointSpec | None = None, )#

Configuration for a speech-to-text model role.

adapter: AdapterSpec#: Model-specific request and response behavior.

endpoint: EndpointSpec#: Endpoint connectivity, authentication, and readiness settings.

deployment: DeploymentSpec#: Launcher ownership metadata for the serving process.

class xr_ai_models.TTSSpec( kind: ModelKind = KIND_OPENAI_COMPAT, base_url: str = '', api_key_env: str | None = None, timeout: float = 30.0, health_check: bool = True, deployment: DeploymentSpec | None = None, *, adapter: AdapterSpec | None = None, endpoint: EndpointSpec | None = None, )#

Configuration for a text-to-speech model role.

adapter: AdapterSpec#: Model-specific request and response behavior.

endpoint: EndpointSpec#: Endpoint connectivity, authentication, and readiness settings.

deployment: DeploymentSpec#: Launcher ownership metadata for the serving process.

class xr_ai_models.VLMSpec( kind: ModelKind = KIND_OPENAI_COMPAT, base_url: str = '', model_name: str = '', api_key_env: str | None = None, capabilities: dict[str, Any] | None = None, default_extras: dict[str, Any] | None = None, timeout: float = 60.0, health_check: bool = True, deployment: DeploymentSpec | None = None, *, adapter: AdapterSpec | None = None, endpoint: EndpointSpec | None = None, thinking: xr_ai_models.ThinkingSpec | None = None, )#

Configuration for an image- or video-language model role.

adapter: AdapterSpec#: Model-specific request and response behavior.

endpoint: EndpointSpec#: Endpoint connectivity, authentication, and readiness settings.

deployment: DeploymentSpec#: Launcher ownership metadata for the serving process.

xr_ai_models.load_models_config(path: pathlib.Path | str) → ModelsConfig#

Load a YAML or JSON model profile and resolve its adapter presets.

The file may contain model entries directly or nested below a models key. Invalid entries raise ValueError with the file path and role name included in the message.

xr_ai_models.load_models_config_from_dict( raw: dict[str, Any], *, source: str = '<dict>', ) → ModelsConfig#

Build a model configuration from a parsed direct or models mapping.

source labels validation errors and is useful when the mapping did not originate from a file.

xr_ai_models.ThinkingControl#: How a model receives the reasoning switch.

class xr_ai_models.ThinkingSpec#

How a model endpoint switches reasoning, and its default state.

control: ThinkingControl | None = None#

The model’s switch dialect.

chat_template sends chat_template_kwargs.enable_thinking, as vLLM and NIM Nemotron/Qwen-style templates expect, and a budget as vLLM’s thinking_token_budget. system_prompt puts a detailed thinking on|off line at the start of the system message, as Llama-Nemotron expects. none leaves requests unchanged for models without a switch or endpoints that reject template arguments. None (unset) uses chat_template when default is set; otherwise only a call that enables thinking sends the switch, and an explicit off adds nothing to the request.

default: bool | None = None#

Reasoning state when a call leaves enable_thinking unset.

None sends no switch, so the model’s own template default applies.

classmethod from_mapping(value: Mapping[str, Any]) → ThinkingSpec#

Build a spec from a thinking configuration mapping.

Raises ValueError for unknown keys or invalid values.

xr_ai_models.make_decision( config: xr_ai_models.ModelsConfig, name: str, ) → xr_ai_models.DecisionService#

Construct the typed decision service for the named configuration entry.