Counters

Counter annotations describe quantities that change over time, such as memory usage, queue depth, bytes processed, or model training metrics.

This page covers counter annotations captured while the application runs. To submit pre-collected samples with explicit timestamps, see Writing Pre-Collected Data.

Create counters with nvtx.Domain.get_counter() and keep the counter object around; call nvtx.Counter.sample() from performance-sensitive code.

Scalar Counters

Use int for signed 64-bit integer samples and float for double-precision floating point samples:

import nvtx
import torch


domain = nvtx.get_domain("Example")
gpu_utilization_counter = domain.get_counter(
    "gpu utilization",
    int,
    description="Percent of time kernels were executing on the GPU",
)

gpu_utilization_counter.sample(torch.cuda.utilization())

Counter Groups

Use a NumPy dtype to expose multiple fields as one counter sample. Structured dtypes represent counter groups:

import nvtx
import torch


domain = nvtx.get_domain("CUDA")
device = torch.cuda.current_device()

gpu_metrics_dtype = nvtx.numpy_dtype([
    ("gpu_utilization", int),
    ("memory_allocated", int),
    ("memory_reserved", int),
])

gpu_metrics_counter = domain.get_counter(
    "gpu metrics",
    gpu_metrics_dtype,
    description="GPU utilization and PyTorch CUDA memory usage",
)

gpu_metrics_counter.sample((
    torch.cuda.utilization(device),
    torch.cuda.memory_allocated(device),
    torch.cuda.memory_reserved(device),
))

Counter groups are flat: fields must be scalar dtypes. Fixed-size array fields and structured or nested fields are not supported.

Avoiding Copies

Counter samples and groups follow the general guidance in Pass data in its native form: pass native Python values as-is, and prefer a C-contiguous NumPy array matching the counter’s dtype when you assemble a batch yourself with nvtx.Counter.batch_submit().

Counter Semantics

Use nvtx.CounterSemantics to describe how counter values should be interpreted, including units, bounds, value type, or interpolation. For top-level scalar counters, pass nvtx.CounterSemantics to nvtx.Domain.get_counter():

gpu_utilization_counter = domain.get_counter(
    "gpu utilization",
    int,
    semantics=nvtx.CounterSemantics(unit="percent", min=0, max=100),
)

For per-field semantics in a counter group, build the field dtype with nvtx.numpy_dtype():

percent_dtype = nvtx.numpy_dtype(
    int,
    counter_semantics=nvtx.CounterSemantics(unit="percent", min=0, max=100),
)
bytes_dtype = nvtx.numpy_dtype(
    int,
    counter_semantics=nvtx.CounterSemantics(unit="bytes", min=0),
)
gpu_metrics_dtype = nvtx.numpy_dtype([
    ("gpu_utilization", percent_dtype),
    ("memory_allocated", bytes_dtype),
    ("memory_reserved", bytes_dtype),
])

Batched Samples

Batched samples can reduce overhead when metrics are produced in a hot path but do not need to be submitted immediately. Store the metric values and timestamps with minimal work in the hot path, then use nvtx.Counter.batch_submit() to submit them together. Use nvtx.Domain.get_timestamp() to get NVTX timestamps:

loss_counter = domain.get_counter("training loss", float)
losses = []
timestamps = []

for inputs, targets in dataloader:
    optimizer.zero_grad(set_to_none=True)
    outputs = model(inputs)
    loss = criterion(outputs, targets)
    loss.backward()
    optimizer.step()

    losses.append(loss.detach())
    timestamps.append(domain.get_timestamp())

loss_samples = [loss.item() for loss in losses]
loss_counter.batch_submit(loss_samples, timestamps)

The default time_domain matches timestamps returned by nvtx.Domain.get_timestamp(). If timestamps come from another clock, create the counter with the matching nvtx.TimestampType so the batch declares the clock domain used by its timestamps.

No-Value Samples

Use nvtx.Counter.sample_no_value() when a sample is known to be zero, unchanged, or unavailable, but no explicit value should be submitted:

gpu_utilization_counter.sample_no_value(nvtx.CounterNoValueReason.UNAVAILABLE)