Installation

We provide pre-built installers for CUDA-Q Realtime that can be downloaded from our GitHub Releases, starting with the CUDA-Q release tag 0.14.0. The binaries are available for x86_64, and ARM64 Linux system with CUDA Versions 12.6+ (Driver 560.35.05+) and 13.x (Driver 580.65.06+). Instructions for building CUDA-Q Realtime from source can be found on our GitHub repository.

CUDA-Q Realtime has been tested with the following NVIDIA products: - NVIDIA IGX Thor - NVIDIA GB200 - NVIDIA DGX Spark

Prerequisites

The network transport bridge for cudaq-realtime is provided by the NVQLink reference architecture, based on the Holoscan Sensor Bridge FPGA IP. Users may also implement their own transport layer and make it accessible to cudaq-realtime as a bridge implementation.

Note

Please make sure doca-sdk-gpunetio is installed along with doca-all. CPU RoCE uses libibverbs instead of DOCA. HSB playback requires the CUDA driver library (libcuda.so.1) with either bridge.

  • CUDA Runtime with version 12.6+ or 13.x

HSB FPGA IP core and RFSoC bit-file

We offer two artifacts to developers seeking to use cudaq-realtime with their FPGA systems:

  1. The open-source HSB FPGA IP core, nv_hsb_ip, can be incorporated in a developer’s FPGA design to enable that design to define an NVQLink-connected device. This core is available as RTL in the HSB FPGA IP core source.

  2. The RFSoC PYNQ example is a fully packaged example for the Real Digital RFSoC 4x2 evaluation board, using Vivado part xczu48dr-ffvg1517-2-e. Use the RFSoC PYNQ source if you want to build or modify the design in Vivado. Use the pre-built nvqlink_rfsoc_v2603.bit bit-file in the HSB 2.6.0-EA artifact directory if you want to program the supported board without rebuilding it.

When building the RFSoC project from source, use the released fpga tree from the Holoscan Sensor Bridge repository, including both fpga/pynq and the sibling fpga/nv_hsb_ip directory. Do not mix nv_hsb_ip or RFSoC PYNQ files from an older HSB release with the packaged RFSoC example. For another RFSoC part or board, update the Vivado part and constraints in fpga/pynq/rfsoc-pynq/build/build.tcl and rebuild the bit-file.

Setup

  • Install CUDA-Q Realtime. For example,

    ./install_cuda_quantum_realtime_cu13.arm64  --accept
    

    The default installation location is /opt/nvidia/cudaq/realtime, which usually requires sudo. To install somewhere else, pass --installpath to the installer. For example,

    ./install_cuda_quantum_realtime_cu13.arm64 --accept -- --installpath $HOME/.cudaq_realtime
    
  • Follow the instructions given by the installer for post-installation steps to set environment variables.

  • Program the FPGA with HSB. See HSB FPGA IP core and RFSoC bit-file for the reusable nv_hsb_ip RTL source, the RFSoC PYNQ source example, and the pre-built RFSoC bit-file.

CUDA-Q Realtime defines a Network Provider Interface to build a networking-agnostic application. This interface consists of a set of APIs to construct a real-time RPC dispatch solution in a networking-agnostic manner. These APIs are backed by a provider plugin (a shared library) implementing the specific transport protocol.

A guide that explains how to integrate a new networking provider with CUDA-Q Realtime via the Network Provider Interface can be found here.

Latency Measurement

The CUDA-Q Realtime installer contains a validation script that includes a latency measurement. After completing the installation and setup steps, you should find the validate.sh script in the CUDA-Q Realtime installation location (usually in /opt/nvidia/cudaq/realtime). The script executes a simple RPC dispatch tests, whereby the FPGA sends data (array of bytes) to the GPU, the GPU performs a simple increment by one calculation on each of the byte in the incoming array (unless in the --forward mode) and returns the array to the FPGA. The validation includes checking the data correctness and measuring the round-trip latency.

The validation script has three defined kernel modes, one for a unified kernel (enabled with --dispatch=unified), one for a forward kernel (enabled with --dispatch=forward), and a 3-kernel setup (used by default). The forward kernel skips the RPC callback to measure the raw latency without any compute performed on the GPU.

bash validate.sh --dispatch=unified \
  --bridge-arg="--device=mlx5_0 --peer-ip=192.168.0.2"

Note

The command line arguments need to be adjusted based on the system setup:

  • --device is the IB device name that is connected to the FPGA.

  • --peer-ip is the IP address of the HSB FPGA.

  • --gpu-id N selects the GPU device (default: 0); --cpu selects CPU dispatch.

  • --dispatch=unified or --dispatch=forward enables unified dispatch or native forwarding.

RoCE with GPU execution is the default. For a local UDP check without an FPGA, run bash validate.sh --udp.

Upon successful completion, the above validation script should print out something similar to the following (latency records abbreviated):

REPLIES PASS: 100/100 packets received correctly
CLOCK tick_period_numerator=1 tick_period_denominator=1000000000 tick_period_seconds=1e-09
LATENCY request_id=1 ticks=6348 ns=6348
...
LATENCY request_id=100 ticks=3589 ns=3589

=== PTP Round-Trip Latency ===
  Samples:  100
  Min:      3589 ns
  Max:      6348 ns
  Avg:      3872.0 ns
  Median:   3800 ns
PASS: 100/100 replies; latency_samples=100/100; provider=libcudaq-playback-provider-hsb.so
VALIDATION PASS execution=gpu dispatch=unified samples=100

Latency statistics are in nanoseconds; the clock time base is in seconds per tick. Unavailable timing is reported separately and does not fail packet validation.

To measure latency with a custom networking implementation, implement a playback provider plugin that sends data to CUDA-Q Realtime according to the custom networking protocol.

For example, in the HSB-based implementation, we use the ptp_timestamp field in the RPCHeader / RPCResponse (see the message protocol documentation) to capture the timestamp for latency analysis. Specifically, the stimulus tool (FPGA) stores the ‘send’ timestamp in the RPCHeader (incoming message), which will be echoed by the GPU in the outgoing RPCResponse after processing it (e.g., with the RPC handler). Using the Integrated Logic Analyzer timestamp when the FPGA receives the response from the GPU, we can compute the round-trip latency. This file contains an example of such a playback provider.