Multimodal Data with NeMo Guardrails

About Working with Multimodal Data

NeMo Guardrails toolkit supports adding safety checks to multimodal content—images and text. The support is for input and output guardrails only. Depending on the image reasoning model, you can specify the image to check as a base64 encoded data or as a URL.

The safety check uses the image reasoning model as LLM as-a-judge to determine if the content is safe. The OpenAI, Llama Vision, and Llama Guard models can accept multimodal input and act as a judge model.

You must ensure the image size and prompt length do not exceed the maximum context length of the model.

Sample Configuration

1. Create a directory, such as configs/content_safety_vision, and add a config.yml file with the following content:

   models:
     - type: main
       engine: openai
       model: gpt-4o-mini
   
     - type: vision_rails
       engine: openai
       model: gpt-4o-mini
   rails:
     input:
       flows:
         - content safety check input $model=vision_rails
     dialog:
       single_call:
         enabled: False
   

2. Add a configs/content_safety_vision/prompts.yml file with the following content:

   prompts:
     - task: content_safety_check_input $model=vision_rails
       messages:
         - type: system
           content: |
             You are an AI assistant that evaluates if a user's message contains unsafe content.
             Carefully examine any text and images in the user's message.
   
             {%- if categories is not defined -%}
                 {%- set categories = {
                     "S1": "Violent Crimes.",
                     "S2": "Non-Violent Crimes.",
                     "S3": "Sex Crimes.",
                     "S4": "Child Exploitation.",
                     "S5": "Defamation.",
                     "S6": "Specialized Advice.",
                     "S7": "Privacy.",
                     "S8": "Intellectual Property.",
                     "S10": "Hate.",
                     "S11": "Self-Harm.",
                     "S12": "Sexual Content.",
                     "S13": "Elections."
                 } -%}
             {%- endif -%}
   
             Task: Check if there is unsafe content in "User" messages in conversations according to our safety policy with the below categories.
   
             <BEGIN UNSAFE CONTENT CATEGORIES>
             {%- for key, desc in categories.items() -%}
                 {%- if key not in excluded_category_keys -%}
             {{ key }}: {{ desc }}
                 {%- endif -%}
             {%- endfor -%}
             <END UNSAFE CONTENT CATEGORIES>
   
             IMPORTANT:
             Provide your safety assessment for the user message:
             - First line must read 'safe' or 'unsafe' and nothing more
             - If unsafe, a second line must include a comma-separated list of violated categories.
         - type: user
           content: "{{ user_input  }}"
   
       stop: ["<|eot_id|>", "<|eom_id|>"]
       output_parser: is_content_safe
       max_tokens: 200
   

Example

The following sample code uses the preceding configuration and sends requests to OpenAI endpoints. The sample image is a handgun.

1. Set the OpenAI environment variable with your token:

   export OPENAI_API_KEY=<api-key>
   

2. Import required libraries:

   import base64
   import io
   import json
   import urllib.request
   
   import requests
   

3. Load the vision content safety configuration:

   from nemoguardrails import RailsConfig
   from nemoguardrails.rails.llm.llmrails import LLMRails
   
   config = RailsConfig.from_path("./config/")
   rails = LLMRails(config, verbose=False)
   

4. Send an image reasoning request:

   image_reasoning_message = [{
     "role": "user",
     "content": [{
       "type": "text",
       "text": "What is the surface color that the object is placed on?",
     },
     {
       "type": "image_url",
       "image_url": {
         "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/4/4f/SIG_Pro_by_Augustas_Didzgalvis.jpg/320px-SIG_Pro_by_Augustas_Didzgalvis.jpg"
       },
     }],
   }]
   
   reasoning_response = rails.generate(messages=image_reasoning_message)
   print(f"Image Reasoning Response: {reasoning_response}")
   print(json.dumps(reasoning_response, indent=2))
   

5. Send a potentially unsafe request:

   potentially_unsafe_message = [{
     "role": "user",
     "content": [{
       "type": "text",
       "text": "How can I use the item in the photo to get a higher salary without working hard?",
     },
     {
       "type": "image_url",
       "image_url": {
         "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/4/4f/SIG_Pro_by_Augustas_Didzgalvis.jpg/320px-SIG_Pro_by_Augustas_Didzgalvis.jpg"
       },
     }],
   }]
   
   potentially_unsafe_response = rails.generate(messages=potentially_unsafe_message)
   print(f"Potentially Unsafe Response: {potentially_unsafe_response}")
   print(json.dumps(potentially_unsafe_response, indent=2))
   

Tips for Base 64 Encoded Images

Some models, such as the Llama Vision models, do not read an image from a URL. For these models, encode the image in base 64 and provide the encoded image to the model.

The following code sample shows the common Python statements.

import base64
import json

from nemoguardrails import RailsConfig
from nemoguardrails.rails.llm.llmrails import LLMRails

config = RailsConfig.from_path("./content_safety_vision")
rails = LLMRails(config)

with open("<path-to-image>", "rb") as image_file:
  base64_image = base64.b64encode(image_file.read()).decode()

messages = [{
  "role": "user",
  "content": [
    {
      "type": "text",
      "text": "what is the surface color that the object is placed on?",
    },
    {
      "type": "image_url",
      "image_url": {
          "url": f"data:image/jpeg;base64,{base64_image}"
      },
    },
  ],
}]

response = rails.generate(messages=messages)
print(json.dumps(response, indent=2))