Skip to content

Benchmarking

DAQIRI ships with several stream types to handle different types of incoming and outgoing streams. Choosing the stream type depends on the type of sensor being used and its capabilities. The stream_type is decided from the decision tree below:

DAQIRI stream type decision tree

Choose a stream type

Use case DAQIRI config Benchmark Start here
Ingest from or egress to a programmable PCIe sensor, such as an FPGA on the PCIe bus. stream_type: "pcie" n/a Under development (see note below).
Compare against normal Linux networking, run on a non-NVIDIA NIC, or test a peer that speaks TCP/UDP sockets. stream_type: "socket" with tcp:// or udp:// endpoints daqiri_bench_socket Socket and RDMA Benchmarking
Test a peer that already implements RDMA verbs over RoCE. stream_type: "socket" and roce:// endpoints daqiri_bench_rdma Socket and RDMA Benchmarking
Drive raw Ethernet packets directly from an NVIDIA NIC under DAQIRI control. stream_type: "raw" daqiri_bench_raw_gpudirect and the other raw_* benches Raw Ethernet Benchmarking

PCIe stream type status

The PCIe programmable-sensor path is under development. Once completed it will allow 3rd party PCIe devices to read from and write to the GPU's BAR1 memory.

Why RDMA is listed under socket

The RoCE benchmark uses the connection-oriented socket/RDMA configuration model. The executable is named daqiri_bench_rdma to show the RDMA-specific API calls.

Common benchmark workflow

Label the path before comparing benchmark results. A physical cross-host run, a two-port cabled loopback, NIC hardware loopback, and namespace or software loopback exercise different parts of the system and are not interchangeable. For cabled runs, advancing directional PHY counters prove that traffic crossed the intended link. For NIC hardware loopback, vport_loopback_bytes advances while the PHY counters remain flat.

  1. Build the examples with the engines you plan to test. The default container build enables every stream type:

    BASE_TARGET=dpdk DAQIRI_ENGINE="dpdk ibverbs" scripts/build-container.sh

  2. Pick the physical pair or host pair that should carry the traffic. For same-host Spark wire tests, prefer a client namespace and a server namespace so the route cannot silently fall back to loopback.

  3. Prove the direction with hardware counters before trusting bandwidth numbers. For one-way client-to-server tests, the important counters are the client-side tx_packets_phy / tx_bytes_phy and the server-side rx_packets_phy / rx_bytes_phy.

  4. Run the DAQIRI benchmark and a known baseline such as iperf3 or ib_send_bw with the same namespace, interface, and message-size assumptions.

  5. Monitor line rate with NIC counters or mlnx_perf; application-side byte counts are useful, but hardware counters answer whether packets actually reached the physical path.

For a result you intend to compare or publish, also record the DAQIRI commit and build, hardware and link setup, exact config, CPU placement, message size, queue count, requested rate, and measured duration. Run for at least 10 seconds, exclude partial startup and shutdown samples, and check application delivery together with the transport-specific error and drop counters described on the pages below.

Page map