Exchange#
cuda.coop.cutlass.exchange() rearranges register values across a group. It
returns a new payload with the requested ownership and preserves the input
payload. Exchange itself performs no global-memory load or store.
The API supports blocked/striped, warp-striped, and scatter modes. Use the
mode and payload form supported by the selected block or warp group; see
cuda.coop.cutlass.exchange().
Striped-to-blocked exchange with eight illustrative threads and two items per thread.#
Values retain their logical sequence while their owning threads change.
Thread 0 1 2 3 4 5 6 7
Striped input 0,8 1,9 2,10 3,11 4,12 5,13 6,14 7,15
Blocked result 0,1 2,3 4,5 6,7 8,9 10,11 12,13 14,15
Reading the stages#
See blocked versus striped for a compact ownership table and the index formulas used here.
The first row groups inputs by thread. The middle row shows shared scratch at logical positions; the final row groups the returned values by their new owners. Select a value to inspect its input and result slots.
For K items per thread and T threads, blocked ownership places
logical value p in thread p // K, slot p % K. Striped ownership
places it in thread p % T, slot p // T. The two blocked/striped modes convert
between these arrangements without changing the logical order.
blocked_to_warp_striped and warp_striped_to_blocked apply that
conversion separately within each physical warp’s tile. The explorer uses
four lanes per teaching warp to fit two warp tiles on screen. Physical
CUDA warps have 32 lanes. Warp-striped ownership differs from stripes
across a block containing multiple warps.
The scatter modes take a signed-integer ranks payload with the same
item count as the input. Each rank selects a logical result position;
scatter_to_blocked and scatter_to_striped expose those destinations
in their named layouts. The explorer uses rank[p] = (5 * p) % N, a
permutation for the illustrated power-of-two tile sizes. Active ranks must
be in range and unique.
scatter_to_striped_guarded suppresses inputs with negative ranks; it
does not guard against ranks beyond the tile extent. The example assigns
rank -1 to every fifth input. scatter_to_striped_flagged instead
uses a non-boolean integer valid_flags payload and suppresses every
fourth input in the example. Suppressed writes leave holes: a question
mark means an unspecified output that must not be consumed, not a zero
or a preserved input value.
The illustrations use the default non-timesliced exchange. The
block API also provides warp_time_slicing for the layout conversions
and unguarded scatters. Guarded and flagged scatter do not support it.
Using Exchange in a kernel#
This CuTe fragment uses the kernel-fragment
setup. Launch with 128 threads and provide at
least 128 * items_per_thread source and destination elements for each
block.
block = coop.this_block()
offset = block_index * 128 * items_per_thread
items = coop.load(
block, source, items_per_thread=items_per_thread,
algorithm="striped", offset=offset,
)
blocked = coop.exchange(block, items, mode="striped_to_blocked")
coop.store(block, destination, blocked, algorithm="direct", offset=offset)
# blocked owns consecutive items; items retains striped ownership.
import cutlass
from cutlass import cute
import cuda.coop.cutlass as cutlass_coop
@cute.kernel
def reverse_tile(
source: cute.Pointer,
destination: cute.Pointer,
items_per_thread: cutlass.Constexpr,
):
block = cutlass_coop.this_block()
thread = block.rank()
items = cutlass_coop.load(
block, source, items_per_thread=items_per_thread
)
ranks = cutlass_coop.ThreadData(items_per_thread)
for item in cutlass.range_constexpr(items_per_thread):
ranks[item] = cutlass.Int32(
64 * items_per_thread - 1 - (thread * items_per_thread + item)
)
reversed_items = cutlass_coop.exchange(
block, items, mode="scatter_to_blocked", ranks=ranks
)
cutlass_coop.store(block, destination, reversed_items)
@cute.jit
def launch(
source: cute.Pointer,
destination: cute.Pointer,
items_per_thread: cutlass.Constexpr,
):
reverse_tile(source, destination, items_per_thread).launch(
grid=1, block=64
)
See cuda.coop.cutlass.exchange() and
the Programming Guide for group participation,
scatter controls, and register payload forms.