mod state#

module state#

Process-global runtime state and middleware-chain builders.

NemoFlowContextState owns the registries and helper methods that power the public scope, tool, and LLM APIs. Advanced integrations can use this type directly to register middleware, attach runtime extensions, and build the resolved callback chains that the higher-level API layer executes.

Structs and Unions

struct NemoFlowContextState#

Process-global runtime state backing middleware and event emission.

The public API layer stores one shared instance of this type for the process. It contains global middleware registries, lifecycle subscribers, and arbitrary extension slots used by bindings or integrations.

tool_sanitize_request_guardrails: SortedRegistry<GuardrailEntry<ToolSanitizeFn>>#

Global tool request sanitizers applied to emitted tool-start payloads.

tool_sanitize_response_guardrails: SortedRegistry<GuardrailEntry<ToolSanitizeFn>>#

Global tool response sanitizers applied to emitted tool-end payloads.

tool_conditional_execution_guardrails: SortedRegistry<GuardrailEntry<ToolConditionalFn>>#

Global tool guardrails that can reject execution before the callback runs.

tool_request_intercepts: SortedRegistry<Intercept<ToolInterceptFn>>#

Global tool request intercepts that can rewrite arguments before execution.

tool_execution_intercepts: SortedRegistry<ExecutionIntercept<ToolExecutionFn>>#

Global tool execution intercepts that wrap or replace callback execution.

llm_sanitize_request_guardrails: SortedRegistry<GuardrailEntry<LlmSanitizeRequestFn>>#

Global LLM request sanitizers applied to emitted LLM-start payloads.

llm_sanitize_response_guardrails: SortedRegistry<GuardrailEntry<LlmSanitizeResponseFn>>#

Global LLM response sanitizers applied to emitted LLM-end payloads.

llm_conditional_execution_guardrails: SortedRegistry<GuardrailEntry<LlmConditionalFn>>#

Global LLM guardrails that can reject execution before the provider callback runs.

llm_request_intercepts: SortedRegistry<Intercept<LlmRequestInterceptFn>>#

Global LLM request intercepts that can rewrite or annotate requests.

llm_execution_intercepts: SortedRegistry<ExecutionIntercept<LlmExecutionFn>>#

Global non-streaming LLM execution intercepts that wrap callback execution.

llm_stream_execution_intercepts: SortedRegistry<ExecutionIntercept<LlmStreamExecutionFn>>#

Global streaming LLM execution intercepts that wrap stream-producing callbacks.

event_subscribers: HashMap<String, EventSubscriberFn>#

Global lifecycle subscribers notified after runtime events are emitted.

extensions: HashMap<String, Box<dyn Any + Send + Sync>>#

Arbitrary binding- or integration-specific runtime extensions.

Implementations

impl NemoFlowContextState#

Functions

fn build_llm_end_event(&self, params: EndLlmHandleParams<'_>) -> Event#

Build an LLM-end event from builder parameters.

The metadata payload is merged over the metadata already stored on the handle.

Parameters

  • params: LLM end-event builder parameters.

Returns

An LLM-end Event derived from the provided parameters.

fn build_llm_start_event(&self, handle: &LlmHandle, data: Option<Json>, annotated_request: Option<Arc<AnnotatedLlmRequest>>) -> Event#

Build an LLM-start event from a handle.

Parameters

  • handle: LLM handle to serialize into an event.

  • data: Sanitized LLM request payload.

  • annotated_request: Optional normalized request annotation.

Returns

An LLM-start Event derived from the provided handle.

fn build_scope_end_event(&self, params: EndScopeHandleParams<'_>) -> Event#

Build a scope-end event from builder parameters.

Parameters

  • params: Scope end-event builder parameters.

Returns

A scope-end Event derived from the provided parameters.

fn build_scope_start_event(&self, handle: &ScopeHandle, data: Option<Json>) -> Event#

Build a scope-start event from a handle.

Parameters

  • handle: Scope handle to serialize into an event.

  • data: Optional semantic input payload exported on the start event.

Returns

A scope-start Event derived from the provided handle.

fn build_tool_end_event(&self, params: EndToolHandleParams<'_>) -> Event#

Build a tool-end event from builder parameters.

The metadata payload is merged over the metadata already stored on the handle.

Parameters

  • params: Tool end-event builder parameters.

Returns

A tool-end Event derived from the provided parameters.

fn build_tool_start_event(&self, handle: &ToolHandle, data: Option<Json>) -> Event#

Build a tool-start event from a handle.

Parameters

  • handle: Tool handle to serialize into an event.

  • data: Optional tool input payload.

Returns

A tool-start Event derived from the provided handle.

fn collect_event_subscribers(&self, scope_local_subscribers: &[EventSubscriberFn]) -> Vec<EventSubscriberFn>#

Combine global and scope-local subscribers into one delivery list.

Parameters

  • scope_local_subscribers: Subscribers collected from the active scope stack.

Returns

A vector containing all global subscribers followed by the provided scope-local subscribers.

fn create_event(&self, params: MarkEvent) -> Event#

Build a standalone mark event.

Parameters

Returns

A mark Event containing the provided MarkEvent.

fn create_llm_handle(&self, params: CreateLlmHandleParams<'_>) -> LlmHandle#

Create a new LLM handle.

Parameters

  • name: Logical provider or model family name.

  • parent_uuid: Optional parent scope UUID.

  • attributes: LLM attribute bitflags.

  • data: Optional application payload stored on the handle.

  • metadata: Optional metadata stored on the handle.

  • model_name: Optional normalized model name stored on the handle.

  • timestamp: Optional handle start time. When omitted, the current UTC time is used.

Returns

A new LlmHandle with a fresh UUID.

fn create_scope_handle(&self, params: CreateScopeHandleParams<'_>) -> ScopeHandle#

Create a new scope handle.

Parameters

  • name: Human-readable scope name.

  • parent_uuid: Optional parent scope UUID.

  • scope_type: Semantic category of the scope.

  • attributes: Scope attribute bitflags.

  • data: Optional application payload stored on the handle.

  • metadata: Optional metadata stored on the handle.

  • timestamp: Optional handle start time. When omitted, the current UTC time is used.

Returns

A new ScopeHandle with a fresh UUID.

fn create_tool_handle(&self, params: CreateToolHandleParams<'_>) -> ToolHandle#

Create a new tool handle.

Parameters

  • name: Tool name recorded on emitted events.

  • parent_uuid: Optional parent scope UUID.

  • attributes: Tool attribute bitflags.

  • data: Optional application payload stored on the handle.

  • metadata: Optional metadata stored on the handle.

  • tool_call_id: Optional provider-specific correlation identifier.

  • timestamp: Optional handle start time. When omitted, the current UTC time is used.

Returns

A new ToolHandle with a fresh UUID.

fn emit_event(event: &Event, subscribers: &[EventSubscriberFn])#

Deliver an event to every subscriber in order.

Parameters

  • event: Fully constructed lifecycle event to deliver.

  • subscribers: Subscribers that should observe the event.

fn end_llm_handle(&self, handle: &LlmHandle, data: Option<Json>, metadata: Option<Json>, annotated_response: Option<Arc<AnnotatedLlmResponse>>) -> Event#

Build an LLM-end event from a handle and optional overrides.

Parameters

  • handle: LLM handle to serialize into an event.

  • data: Sanitized LLM response payload.

  • metadata: Optional metadata payload merged over handle.metadata.

  • annotated_response: Optional normalized response annotation.

Returns

An LLM-end Event derived from the provided handle.

fn end_scope_handle(&self, handle: &ScopeHandle, data: Option<Json>) -> Event#

Build a scope-end event from a handle.

Parameters

  • handle: Scope handle to serialize into an event.

  • data: Optional data payload returned from the scope.

Returns

A scope-end Event derived from the provided handle.

fn end_tool_handle(&self, handle: &ToolHandle, data: Option<Json>, metadata: Option<Json>) -> Event#

Build a tool-end event from a handle and optional overrides.

Parameters

  • handle: Tool handle to serialize into an event.

  • data: Optional end-event data payload.

  • metadata: Optional metadata payload merged over handle.metadata.

Returns

A tool-end Event derived from the provided handle.

fn get_extension<T: Any + Send + Sync>(&self, key: &str) -> Option<&T>#

Borrow a typed runtime extension by key.

Parameters

  • key: Extension slot name.

Returns

Some(&T) when an extension exists under key with the requested type and None otherwise.

fn get_extension_mut<T: Any + Send + Sync>(&mut self, key: &str) -> Option<&mut T>#

Mutably borrow a typed runtime extension by key.

Parameters

  • key: Extension slot name.

Returns

Some(&mut T) when an extension exists under key with the requested type and None otherwise.

fn llm_build_execution_chain(&self, name: &str, default_fn: LlmExecutionNextFn, scope_locals: &[&SortedRegistry<ExecutionIntercept<LlmExecutionFn>>]) -> LlmExecutionNextFn#

Build the composed non-streaming LLM execution continuation chain.

Parameters

  • name: Logical provider or model family name passed into each execution intercept.

  • default_fn: Base provider callback that should run after all intercepts.

  • scope_locals: Scope-local execution intercept registries collected from the active scope stack.

Returns

A composed LlmExecutionNextFn that wraps default_fn in every matching execution intercept.

fn llm_conditional_execution_chain(&self, request: &LlmRequest, scope_locals: &[&SortedRegistry<GuardrailEntry<LlmConditionalFn>>]) -> crate::error::Result<Option<String>>#

Evaluate LLM conditional-execution guardrails in priority order.

Parameters

  • request: LLM request to validate.

  • scope_locals: Scope-local conditional guardrail registries collected from the active scope stack.

Returns

A Result containing Ok(None) when execution is allowed or Ok(Some(reason)) when a guardrail rejects the call.

Errors

Propagates any error returned by a guardrail callback.

fn llm_request_intercepts_chain(&self, name: &str, request: LlmRequest, annotated: Option<AnnotatedLlmRequest>, scope_locals: &[&SortedRegistry<Intercept<LlmRequestInterceptFn>>]) -> crate::error::Result<(LlmRequest, Option<AnnotatedLlmRequest>)>#

Run LLM request intercepts in priority order.

Parameters

  • name: Logical provider or model family name.

  • request: LLM request to pass through the intercept chain.

  • annotated: Optional normalized request annotation to carry through the chain.

  • scope_locals: Scope-local request intercept registries collected from the active scope stack.

Returns

A Result containing the final request and annotation pair.

Errors

Propagates any error returned by an intercept callback.

Notes

If an intercept entry has break_chain enabled, later intercepts are skipped after that entry runs.

fn llm_sanitize_request_chain(&self, request: LlmRequest, scope_locals: &[&SortedRegistry<GuardrailEntry<LlmSanitizeRequestFn>>]) -> LlmRequest#

Run LLM request sanitizers across global and scope-local registries.

Parameters

  • request: Raw LLM request to sanitize for observability.

  • scope_locals: Scope-local sanitizer registries collected from the active scope stack.

Returns

The sanitized LlmRequest after every matching guardrail has run.

fn llm_sanitize_response_chain(&self, response: Json, scope_locals: &[&SortedRegistry<GuardrailEntry<LlmSanitizeResponseFn>>]) -> Json#

Run LLM response sanitizers across global and scope-local registries.

Parameters

  • response: Raw response payload to sanitize for observability.

  • scope_locals: Scope-local sanitizer registries collected from the active scope stack.

Returns

The sanitized response payload after every matching guardrail has run.

fn llm_stream_build_execution_chain(&self, name: &str, default_fn: LlmStreamExecutionNextFn, scope_locals: LlmStreamExecutionRegistryRefs<'_>) -> LlmStreamExecutionNextFn#

Build the composed streaming LLM execution continuation chain.

Parameters

  • name: Logical provider or model family name passed into each execution intercept.

  • default_fn: Base stream-producing callback that should run after all intercepts.

  • scope_locals: Scope-local execution intercept registries collected from the active scope stack.

Returns

A composed LlmStreamExecutionNextFn that wraps default_fn in every matching execution intercept.

fn new() -> Self#

Create an empty runtime state with no registered middleware.

Returns

A NemoFlowContextState with empty registries, no subscribers, and no extensions.

fn remove_extension(&mut self, key: &str) -> bool#

Remove a runtime extension by key.

Parameters

  • key: Extension slot name.

Returns

true when an extension was removed and false when no extension was stored under key.

fn set_extension<T: Any + Send + Sync>(&mut self, key: impl Into<String>, value: T)#

Store an arbitrary runtime extension under key.

Extensions let bindings or integrations attach shared state to the process-global runtime without adding new first-class fields.

Parameters

  • key: Stable identifier for the extension slot.

  • value: Typed extension value to store.

fn tool_build_execution_chain(&self, name: &str, default_fn: ToolExecutionNextFn, scope_locals: &[&SortedRegistry<ExecutionIntercept<ToolExecutionFn>>]) -> ToolExecutionNextFn#

Build the composed tool execution continuation chain.

Parameters

  • name: Tool name passed into each execution intercept.

  • default_fn: Base tool callback that should run after all intercepts.

  • scope_locals: Scope-local execution intercept registries collected from the active scope stack.

Returns

A composed ToolExecutionNextFn that wraps default_fn in every matching execution intercept.

fn tool_conditional_execution_chain(&self, name: &str, args: &Json, scope_locals: &[&SortedRegistry<GuardrailEntry<ToolConditionalFn>>]) -> crate::error::Result<Option<String>>#

Evaluate tool conditional-execution guardrails in priority order.

Parameters

  • name: Tool name associated with the request.

  • args: Tool arguments to validate.

  • scope_locals: Scope-local conditional guardrail registries collected from the active scope stack.

Returns

A Result containing Ok(None) when execution is allowed or Ok(Some(reason)) when a guardrail rejects the call.

Errors

Propagates any error returned by a guardrail callback.

fn tool_request_intercepts_chain(&self, name: &str, args: Json, scope_locals: &[&SortedRegistry<Intercept<ToolInterceptFn>>]) -> crate::error::Result<Json>#

Run tool request intercepts in priority order.

Parameters

  • name: Tool name associated with the request.

  • args: Tool arguments to pass through the intercept chain.

  • scope_locals: Scope-local request intercept registries collected from the active scope stack.

Returns

A Result containing the final JSON argument payload.

Errors

Propagates any error returned by an intercept callback.

Notes

If an intercept entry has break_chain enabled, later intercepts are skipped after that entry runs.

fn tool_sanitize_request_chain(&self, name: &str, args: Json, scope_locals: &[&SortedRegistry<GuardrailEntry<ToolSanitizeFn>>]) -> Json#

Run tool request sanitizers across global and scope-local registries.

Parameters

  • name: Tool name associated with the request.

  • args: Raw tool arguments to sanitize for observability.

  • scope_locals: Scope-local sanitizer registries collected from the active scope stack.

Returns

The sanitized JSON payload after every matching guardrail has run.

fn tool_sanitize_response_chain(&self, name: &str, result: Json, scope_locals: &[&SortedRegistry<GuardrailEntry<ToolSanitizeFn>>]) -> Json#

Run tool response sanitizers across global and scope-local registries.

Parameters

  • name: Tool name associated with the response.

  • result: Raw tool result to sanitize for observability.

  • scope_locals: Scope-local sanitizer registries collected from the active scope stack.

Returns

The sanitized JSON payload after every matching guardrail has run.

Traits implemented

impl Default for NemoFlowContextState#