Installation¶
We provide pre-built installers for CUDA-Q Realtime that can be downloaded from our
GitHub Releases, starting with the CUDA-Q release tag 0.14.0.
The binaries are available for x86_64, and ARM64 Linux system with CUDA Versions 12.6+ (Driver 560.35.05+) and 13.x (Driver 580.65.06+). Instructions for building CUDA-Q Realtime from source can be found on our GitHub repository.
CUDA-Q Realtime has been tested with the following NVIDIA products: - NVIDIA IGX Thor - NVIDIA GB200 - NVIDIA DGX Spark
Prerequisites¶
The network transport bridge for cudaq-realtime is provided by the NVQLink reference architecture, based on the Holoscan Sensor Bridge FPGA IP.
Users may also implement their own transport layer and make it accessible to cudaq-realtime as a bridge implementation.
CUDA Runtime with version 12.6+ or 13.x
For GPU RoCE: DOCA 3.4.0 installation with
gpunetiosupport.
Note
Please make sure doca-sdk-gpunetio is installed along with doca-all.
CPU RoCE uses libibverbs instead of DOCA. HSB playback requires the
CUDA driver library (libcuda.so.1) with either bridge.
CUDA Runtime with version 12.6+ or 13.x
HSB FPGA IP core and RFSoC bit-file¶
We offer two artifacts to developers seeking to use cudaq-realtime with their FPGA systems:
The open-source HSB FPGA IP core,
nv_hsb_ip, can be incorporated in a developer’s FPGA design to enable that design to define an NVQLink-connected device. This core is available as RTL in the HSB FPGA IP core source.The RFSoC PYNQ example is a fully packaged example for the Real Digital RFSoC 4x2 evaluation board, using Vivado part
xczu48dr-ffvg1517-2-e. Use the RFSoC PYNQ source if you want to build or modify the design in Vivado. Use the pre-builtnvqlink_rfsoc_v2603.bitbit-file in the HSB 2.6.0-EA artifact directory if you want to program the supported board without rebuilding it.
When building the RFSoC project from source, use the released fpga tree from the Holoscan Sensor Bridge repository, including both fpga/pynq and the sibling fpga/nv_hsb_ip directory.
Do not mix nv_hsb_ip or RFSoC PYNQ files from an older HSB release with the packaged RFSoC example.
For another RFSoC part or board, update the Vivado part and constraints in fpga/pynq/rfsoc-pynq/build/build.tcl and rebuild the bit-file.
Setup¶
Install CUDA-Q Realtime. For example,
./install_cuda_quantum_realtime_cu13.arm64 --acceptThe default installation location is
/opt/nvidia/cudaq/realtime, which usually requiressudo. To install somewhere else, pass--installpathto the installer. For example,./install_cuda_quantum_realtime_cu13.arm64 --accept -- --installpath $HOME/.cudaq_realtimeFollow the instructions given by the installer for post-installation steps to set environment variables.
Program the FPGA with HSB. See HSB FPGA IP core and RFSoC bit-file for the reusable
nv_hsb_ipRTL source, the RFSoC PYNQ source example, and the pre-built RFSoC bit-file.
Note
Please make sure to set up the host system and the set up the HSB FPGA device IP address.
CUDA-Q Realtime defines a Network Provider Interface to build a networking-agnostic application. This interface consists of a set of APIs to construct a real-time RPC dispatch solution in a networking-agnostic manner. These APIs are backed by a provider plugin (a shared library) implementing the specific transport protocol.
A guide that explains how to integrate a new networking provider with CUDA-Q Realtime via the Network Provider Interface can be found here.
Latency Measurement¶
The CUDA-Q Realtime installer contains a validation script that includes a latency measurement.
After completing the installation and setup steps, you should find the validate.sh script in the CUDA-Q Realtime installation location
(usually in /opt/nvidia/cudaq/realtime).
The script executes a simple RPC dispatch tests, whereby
the FPGA sends data (array of bytes) to the GPU, the GPU performs
a simple increment by one calculation on each of the byte
in the incoming array (unless in the --forward mode) and returns the array to the FPGA.
The validation includes checking the data correctness and measuring the round-trip latency.
The validation script has three defined kernel modes, one for a unified kernel (enabled with --dispatch=unified), one for a forward kernel
(enabled with --dispatch=forward), and a 3-kernel setup (used by default).
The forward kernel skips the RPC callback to measure the raw latency without any compute performed on the GPU.
bash validate.sh --dispatch=unified \
--bridge-arg="--device=mlx5_0 --peer-ip=192.168.0.2"
Note
The command line arguments need to be adjusted based on the system setup:
--deviceis theIBdevice name that is connected to the FPGA.--peer-ipis the IP address of theHSBFPGA.--gpu-id Nselects the GPU device (default: 0);--cpuselects CPU dispatch.--dispatch=unifiedor--dispatch=forwardenables unified dispatch or native forwarding.
RoCE with GPU execution is the default. For a local UDP check without an FPGA,
run bash validate.sh --udp.
Upon successful completion, the above validation script should print out something similar to the following (latency records abbreviated):
REPLIES PASS: 100/100 packets received correctly
CLOCK tick_period_numerator=1 tick_period_denominator=1000000000 tick_period_seconds=1e-09
LATENCY request_id=1 ticks=6348 ns=6348
...
LATENCY request_id=100 ticks=3589 ns=3589
=== PTP Round-Trip Latency ===
Samples: 100
Min: 3589 ns
Max: 6348 ns
Avg: 3872.0 ns
Median: 3800 ns
PASS: 100/100 replies; latency_samples=100/100; provider=libcudaq-playback-provider-hsb.so
VALIDATION PASS execution=gpu dispatch=unified samples=100
Latency statistics are in nanoseconds; the clock time base is in seconds per tick. Unavailable timing is reported separately and does not fail packet validation.
To measure latency with a custom networking implementation, implement a playback provider plugin that sends data to CUDA-Q Realtime according to the custom networking protocol.
For example, in the HSB-based implementation, we use the ptp_timestamp field in the RPCHeader / RPCResponse (see the message protocol documentation) to capture the timestamp for latency analysis. Specifically, the stimulus tool (FPGA) stores the ‘send’ timestamp in the RPCHeader (incoming message), which will be echoed by the GPU in the outgoing RPCResponse after processing it (e.g., with the RPC handler). Using the Integrated Logic Analyzer timestamp when the FPGA receives the response from the GPU, we can compute the round-trip latency.
This file contains an example of such a playback provider.