xr_ai_voice#

Public voice runtime for XR agents.

Applications configure and register VoiceAgent and may route concurrent producers through VoiceAggregationAgent; media sessions, audio framing, and pipeline processors are implementation details.

Attributes#

`VOICE_CONTRIBUTION_TOPIC`	Candidate spoken output consumed by `VoiceAggregationAgent`.
`VOICE_OUTPUT_TOPIC`	Finite or incremental responses consumed by `VoiceAgent`.
`VOICE_TRANSCRIPT_TOPIC`	Non-empty final STT results queued for publication before gate filtering.

Exceptions#

VoiceStreamClosedError

Raised when output targets a voice stream the consumer already closed.

Classes#

`VoiceAggregationAgent`	Serialize and coalesce candidate speech independently per participant.
`VadConfig`	Tuning knobs for the Silero-VAD utterance detector.
`UserQuery`	One accepted user query emitted by the voice input boundary.
`VoiceAgent`	Own voice media lifecycle and bridge runtime input and output topics.
`VoiceInterrupted`	Notification that participant-scoped or global voice work was interrupted.
`VoiceOutput`	One complete response or one chunk of an incremental voice response.
`VoiceParticipantJoined`	Notification that one participant joined the voice transport.
`VoiceParticipantLeft`	Notification that one participant left the voice transport.
`VoiceTranscript`	One final STT result emitted before voice-gate filtering.
`HubVoiceTransport`	Own the hub endpoint and bidirectional voice media transport.

Package Contents#

xr_ai_voice.VOICE_CONTRIBUTION_TOPIC#: Candidate spoken output consumed by VoiceAggregationAgent.

class xr_ai_voice.VoiceAggregationAgent( *, llm: xr_ai_models.LLMService, prompt: str = _DEFAULT_PROMPT, coalesce_window_s: float = 0.15, queue_capacity: int = 64, max_batch_size: int = 8, max_tokens: int = 192, stream_idle_timeout_s: float = 15.0, participant_idle_timeout_s: float = 60.0, speech_rate_wpm: float = 150.0, minimum_playback_s: float = 0.4, maximum_playback_s: float = 30.0, rewrite_timeout_s: float = 5.0, )#

Serialize and coalesce candidate speech independently per participant.

A lone finite contribution passes through without a model call after the short coalescing window. Completed text is finalized downstream immediately, while the aggregator independently reserves its estimated spoken duration so later updates wait and coalesce instead of building a speech queue. Multiple pending finite contributions are rewritten into one response. A lone incremental response streams through immediately and uses the same scheduling reservation after its final chunk. An urgent contribution interrupts the active output.

async contribute( output: xr_ai_voice.VoiceOutput, ctx: xr_ai_runtime.RuntimeContext, ) → None#: Enqueue one finite response or incremental response fragment.

async stop() → None#: Cancel and await all participant aggregation tasks.

async release(participant_id: str) → None#: Cancel and release one departed participant’s aggregation state.

class xr_ai_voice.VadConfig#

Tuning knobs for the Silero-VAD utterance detector.

Mirrors the constructor of xr_ai_vad.VadDetector. Default values match the in-tree samples’ current behavior.

stop_probe_after_s — cadence in seconds for up to three STT probes of the partial audio buffer. This gives STOP commands a fast interruption path without replacing final transcription, and lets a configured wake phrase be acknowledged before the utterance ends. A phrase-gated utterance can make up to three partial STT requests plus the final request. Set to 0 or negative to disable probes.

silence_duration: float = 0.8#: Seconds of silence that finalize an utterance.

min_speech: float = 0.15#: Minimum speech duration accepted as an utterance.

silero_threshold: float = 0.5#: Silero VAD speech-probability threshold.

stop_probe_after_s: float = 0.25#: Cadence for bounded early wake and STOP transcription probes.

xr_ai_voice.VOICE_OUTPUT_TOPIC#: Finite or incremental responses consumed by VoiceAgent.

xr_ai_voice.VOICE_TRANSCRIPT_TOPIC#

Non-empty final STT results queued for publication before gate filtering.

Delivery is best effort: the bounded queue drops its oldest pending item on overflow, and shutdown discards queued items that have not been published.

class xr_ai_voice.UserQuery#

One accepted user query emitted by the voice input boundary.

text: str#: Gate-accepted speech or direct typed text.

timestamp_us: int#: Input presentation time as Unix microseconds.

class xr_ai_voice.VoiceAgent( *, query_topic: xr_ai_runtime.Topic[UserQuery], stt: xr_ai_models.STTService, tts: xr_ai_models.TTSService, vad: xr_ai_voice.VadConfig, voice_gate: xr_ai_voicegate.VoiceGateConfig, probes: collections.abc.Mapping[str, collections.abc.Callable[[], collections.abc.Awaitable[bool]]] | None = None, ready_file: pathlib.Path | None = None, closeables: collections.abc.Iterable[Any] = (), text_topic: str = 'agent.response', idle_timeout_secs: float | None = None, transport: xr_ai_voice.HubVoiceTransport | None = None, response_capacity: int = 32, text_input: bool = True, participant_joined_topic: xr_ai_runtime.Topic[VoiceParticipantJoined] | None = None, participant_left_topic: xr_ai_runtime.Topic[VoiceParticipantLeft] | None = None, interrupted_topic: xr_ai_runtime.Topic[VoiceInterrupted] | None = None, interrupt_on_supersede: bool = False, )#

Own voice media lifecycle and bridge runtime input and output topics.

Startup waits only for explicitly supplied probes, such as an inference warmup or an application capability dependency. STT and TTS health are not polled automatically. Readiness reports initialized application transport after those explicit probes, not availability of every model endpoint.

async run( runtime: xr_ai_runtime.AgentRuntime, *, source: str = 'voice', ) → None#: Run the owned voice session and bridge it to a running runtime.

async output(output: VoiceOutput, ctx: xr_ai_runtime.RuntimeContext) → None#: Send one voice message using participant and producer metadata.

class xr_ai_voice.VoiceInterrupted#: Notification that participant-scoped or global voice work was interrupted.

class xr_ai_voice.VoiceOutput#

One complete response or one chunk of an incremental voice response.

text: str = ''#: Complete response text or one incremental fragment.

response_id: str | None#: Identifier shared by incremental chunks; recent reuse is ignored with one warning.

final: bool = True#: Whether this message completes the response.

interrupt: bool = False#: Whether the first response message supersedes active voice output.

timestamp_us: int | None#: Optional originating input timestamp propagated to TTS.

validate_boundary() → VoiceOutput#: Require identity for incremental output and text for finite output.

class xr_ai_voice.VoiceParticipantJoined#: Notification that one participant joined the voice transport.

class xr_ai_voice.VoiceParticipantLeft#: Notification that one participant left the voice transport.

exception xr_ai_voice.VoiceStreamClosedError#: Raised when output targets a voice stream the consumer already closed.

class xr_ai_voice.VoiceTranscript#

One final STT result emitted before voice-gate filtering.

text: str#: Final text returned by STT before voice-gate filtering.

timestamp_us: int#: Speech presentation time as Unix microseconds.

class xr_ai_voice.HubVoiceTransport( input_name: str | None = None, output_name: str | None = None, )#

Own the hub endpoint and bidirectional voice media transport.

property endpoint: xr_ai_hub.ProcessorEndpoint#: Return the owned hub processor endpoint.

async wait_until_started() → None#: Wait until the input transport has started its hub IPC receiver.

async send_return_data(msg: xr_ai_hub.DataMessage) → None#: Send participant-routed data through the hub.

property target_participant: str#: Return the participant currently selected for output.

set_target_participant(pid: str) → None#: Select the participant that receives subsequent output.

cleanup_participant(pid: str) → None#: Clear output routing when the selected participant leaves.

shutdown() → None#: Stop and close the owned hub endpoint.