Jailbreak Detection Deployment#

Note

The recommended way to use Jailbreak Detection Heuristics and models with NeMo Guardrails is using the provided Dockerfile. For more details, check out how to build and use the image.

In order to deploy the jailbreak detection server, follow these steps:

  1. Install the dependencies

pip install transformers torch uvicorn nemoguardrails

  1. Start the jailbreak detection server

python -m nemoguardrails.library.jailbreak_detection.server --port 1337

By default, the jailbreak detection server listens on port 1337. You can change the port using the --port option.

Running on GPU#

To run on GPU, ensure you have the NVIDIA Container Toolkit installed. If you are building a container from the provided dockerfiles, make sure that you specify the correct Dockerfile and include the -f parameter with docker build. When running docker, ensure you pass the -e NVIDIA_DRIVER_CAPABILITIES=compute,utility, -e NVIDIA_VISIBLE_DEVICES=all and the --runtime=nvidia argument to docker run.

docker run -ti --runtime=nvidia -e NVIDIA_DRIVER_CAPABILITIES=compute,utility -e NVIDIA_VISIBLE_DEVICES=all <image_name>
On this page