Batched Warp Reduction#

cuda.coop.cutlass.reduce_batched() reduces several independent batches across a warp. Each thread contributes one item to each batch: local slot zero belongs to batch zero, slot one to batch one, and so on. The result has one aggregate per batch, distributed among the threads.

This is useful when every lane holds several features for one sample and the kernel needs a sum, minimum, or maximum for each feature. Ordinary reduce() combines a group’s payload items into one aggregate; batched reduction retains the separate batch axis.

Four lanes each hold three features. Their per-feature sums are 16, 20, and 24, returned in lanes zero, one, and two. Lane three has no result.

Three batches across a logical warp of four lanes.#

Each column below is one independent reduction. ? is an unspecified output slot, which the kernel must not read or store.

Lane        Input slots (batches 0, 1, 2)    Returned slot
L0                     1   2   3                  16
L1                     3   4   5                  20
L2                     5   6   7                  24
L3                     7   8   9                   ?
Batch sums            16  20  24

Input slots and output ownership#

For B batches and a warp of W threads, each input payload contains B items. Each returned payload contains ceil(B / W) slots. output_layout="striped" assigns batch lane + slot * W to each result slot. output_layout="blocked" assigns batch lane * ceil(B / W) + slot instead. The input interpretation is the same for both layouts.

Try ten batches with four lanes to see the distinction: striped lane zero receives batches 0, 4, and 8; blocked lane zero receives batches 0, 1, and 2. The remaining two allocated output slots do not correspond to any batch. Guard stores by batch index, rather than assuming every slot in the returned payload is valid.

The explorer groups inputs by batch to explain the reduction axis. It does not imply that CUB performs a shared-memory transpose. CUB provides a native WarpReduceBatched collective; the diagrams describe its result contract, not its compiled instruction order.

Using batched reduction in a kernel#

This tested example sums items_per_thread features across each physical warp. The input is laid out as 32 consecutive samples per warp, with items_per_thread consecutive features per sample. Each warp writes one result per feature.

@cute.kernel
def feature_sums(
    samples: cute.Pointer,
    totals: cute.Pointer,
    items_per_thread: cutlass.Constexpr,
):
    warp = coop.this_warp()
    features = coop.load(warp, samples, items_per_thread=items_per_thread)
    sums = coop.reduce_batched(warp, features)
    outputs = cute.make_tensor(
        totals, cute.make_layout(2 * items_per_thread)
    )
    # Output slots are striped across the warp lanes.
    warp_index = coop.this_block().rank() // 32
    for item in cutlass.range_constexpr((items_per_thread + 31) // 32):
        feature = warp.rank() + item * 32
        if feature < items_per_thread:
            outputs[warp_index * items_per_thread + feature] = sums[item]

@cute.jit
def launch(
    samples: cute.Pointer,
    totals: cute.Pointer,
    items_per_thread: cutlass.Constexpr,
):
    feature_sums(samples, totals, items_per_thread).launch(
        grid=1, block=(8, 4, 2)
    )

The API supports complete physical warps and logical warps of 1, 2, 4, 8, 16, or 32 threads. Every member of the selected warp must participate, even if that member owns no result. Other logical warps may take another branch. The compiler manages provider storage; this operation does not accept a caller TempStorage. The CUTLASS provider needs no shared scratch allocation or trailing storage-reuse barrier.

The batch count is a positive compile-time payload extent. The result uses the input dtype without accumulator promotion. Built-in operator strings match reduce(). CuTe register tensors and immutable vectors are accepted along with ThreadData. The operation preserves its input and returns a fresh payload. Use only the result slots whose batch index is in range; no temporary-storage descriptor is needed.