NeMo Guardrails Example Plugin#

This example shows how to write a Python NeMo Relay plugin that calls the NeMo Guardrails Python API.

The example lives under examples/nemoguardrails. The single-file plugin implementation, runnable agent, and Guardrails config artifacts are under example. It is not part of the nemo_relay Python package, and NeMo Relay does not depend on nemoguardrails. Applications that use the example install NeMo Guardrails in their own environment and import or vendor the example plugin.

Install#

Install NeMo Relay normally, then install NeMo Guardrails in the application or example environment that activates the plugin:

pip install nemoguardrails

The bundled example config uses NeMo Guardrails’ nvidia_ai_endpoints model engine. Install the NVIDIA LangChain provider when you want to run that config as-is:

pip install langchain-nvidia-ai-endpoints

Configure#

Guardrails stay in native NeMo Guardrails config. Point the plugin at a Guardrails config directory, or pass inline YAML content.

import asyncio

import nemo_relay
import plugin as nemoguardrails_plugin


async def main() -> None:
    nemoguardrails_plugin.register()
    try:
        config = nemo_relay.plugin.PluginConfig(
            components=[
                nemo_relay.plugin.ComponentSpec(
                    kind=nemoguardrails_plugin.DEFAULT_KIND,
                    config={
                        "config_path": "./rails",
                        "codec": "openai_chat",
                    },
                )
            ]
        )
        await nemo_relay.plugin.initialize(config)
    finally:
        nemo_relay.plugin.clear()
        nemoguardrails_plugin.deregister()


asyncio.run(main())

The config_path directory is a normal NeMo Guardrails config directory. For example:

# rails/config.yml
models:
  - type: main
    engine: nvidia_ai_endpoints
    model: meta/llama-3.1-8b-instruct

rails:
  input:
    flows:
      - self check input
  output:
    flows:
      - self check output

prompts:
  - task: self_check_input
    content: |-
      You are checking whether a NeMo Relay request should be allowed.
      The input may be plain user text or a JSON object with tool_name and
      arguments fields.
      User input: {{ user_input }}
      Should this request be blocked? Answer only Yes or No.

  - task: self_check_output
    content: |-
      You are checking whether a NeMo Relay response should be returned.
      The output may be assistant text or a JSON object with tool_name,
      arguments, and result fields.
      Model output: {{ bot_response }}
      Should this response be blocked? Answer only Yes or No.

The plugin config accepts these fields:

  • config_path: Path to a NeMo Guardrails config directory.

  • config_yaml: Inline NeMo Guardrails YAML config.

  • colang_content: Optional inline Colang content. This can only be used with config_yaml.

  • codec: One of openai_chat, openai_responses, or anthropic_messages. This is required when input or output is enabled.

  • input: Whether to run input rails around LLM calls. Defaults to true.

  • output: Whether to run output rails around LLM calls. Defaults to true.

  • tool_input: Whether to check managed tool arguments before execution. Defaults to false.

  • tool_output: Whether to check managed tool results after execution. Defaults to false.

  • priority: Execution-intercept priority. Defaults to 100.

Exactly one of config_path or config_yaml is required.

Example Agent#

The example includes agent_example.py, a concrete example agent that initializes the plugin, checks a managed tools.execute(...) call, and checks a managed llm.execute(...) call against live NVIDIA-hosted inference.

Run it from a checkout where NeMo Relay and NeMo Guardrails are installed. The default lane uses a passthrough Guardrails config and the current_time tool. This is the fastest live validation path because it exercises the real plugin, real nemoguardrails initialization, tool execution, and LLM execution without running model-backed self-check rails:

export NVIDIA_API_KEY="<your-key>"
python examples/nemoguardrails/example/agent_example.py

To run the inline self-check rails example, load example_config.yml from example and pass it as inline config_yaml:

python examples/nemoguardrails/example/agent_example.py --guardrails-config inline

The config directory lane uses the bundled examples/nemoguardrails/example/rails/config.yml by default. It contains the same input and output self-check rails as example/example_config.yml:

python examples/nemoguardrails/example/agent_example.py --guardrails-config path

Use --tool weather when you want the example to use a weather tool instead of the default current_time tool:

python examples/nemoguardrails/example/agent_example.py --tool weather

Pass --config-path when you want the example agent to use your own native NeMo Guardrails config directory:

python examples/nemoguardrails/example/agent_example.py \
  --guardrails-config path \
  --config-path ./rails

Runtime Behavior#

For non-streaming llm.execute(...) calls, the plugin checks the user input before the model call and checks the assistant text after the model call. Guardrails can pass, block, or rewrite input. For output, this example supports pass and block; modified output raises because NeMo Relay response codecs are decode-only and the example does not rewrite provider-shaped responses.

For managed tools.execute(...) calls, the plugin can also check serialized tool arguments before execution and serialized tool results after execution. When Guardrails rewrites tool arguments or results, the rewritten content must be valid JSON.

The bundled config uses the same NeMo Guardrails input and output self-check rails for both LLM messages and tool payloads. The plugin makes tool calls visible to Guardrails by serializing managed tool arguments and results as JSON message content.

This behavior changes the real execution path. It is not an observability-only sanitize guardrail.

Supported Codecs#

The example is intentionally limited to NeMo Relay’s built-in LLM codec shapes:

  • openai_chat for OpenAI Chat Completions-style requests and responses.

  • openai_responses for OpenAI Responses API-style requests and responses.

  • anthropic_messages for Anthropic Messages-style requests and responses.

Provider-specific payloads outside those codecs need a NeMo Relay codec and a response text replacement strategy before a production plugin can apply modified output safely.

Limitations#

This example calls NeMo Guardrails check_async, not generate_async. It checks around NeMo Relay LLM and tool execution calls, but it does not let NeMo Guardrails take over generation or agent orchestration.

The example does not support:

  • Streaming LLM calls.

  • Dialog rails, retrieval rails, execution rails, or generation rails that require NeMo Guardrails to orchestrate the full generation flow.

  • Arbitrary provider payloads beyond the three supported NeMo Relay codecs.

  • Applying modified LLM output back into provider responses.

  • Rewriting tool-call arguments inside model responses before an application turns those model tool calls into managed tools.execute(...) calls.

Tool checks use serialized JSON and NeMo Guardrails input/output checks. They are NeMo Relay tool middleware checks powered by Guardrails, not a full generate_async agent-loop integration.

config_path points at native NeMo Guardrails configuration. Guardrails config can load project code such as actions, so treat that path as trusted application code.