cuda.core.utils.copy_batch#

cuda.core.utils.copy_batch(Stream stream: Stream, srcs: Sequence[Buffer], dsts: Sequence[Buffer], *, options: CopyOptions | Sequence[CopyOptions] | None = None) → None#

Copy a batch of buffers asynchronously.

Source buffer and destination buffer sizes must match. For a single buffer, use Buffer.copy_to() or Buffer.copy_from().

The driver provides no graph-node form of cuMemcpyBatchAsync, so this cannot be captured into a graph. Both passing a GraphBuilder and passing its underlying stream while capture is active are rejected. Build graph copies with graph.GraphNode.memcpy() or per-buffer Buffer.copy_to().

Parameters:

  • stream (Stream) – Stream for the asynchronous copy. First positional and required (mirrors launch()). Does not accept a capturing stream (including a GraphBuilder’s underlying stream); use graph.GraphNode.memcpy() or per-buffer Buffer.copy_to() to build copies into a graph.

  • srcs (Sequence[Buffer]) – Source buffers. Must be a sequence, not a single Buffer.

  • dsts (Sequence[Buffer]) – Destination buffers. Must match len(srcs).

  • options (CopyOptions | Sequence[CopyOptions] | None) – Per-copy options. A single value applies to every copy; a sequence pairs by index and must match len(srcs). None uses stream-ordered defaults.

Raises:

  • ValueError – If lengths or sizes mismatch.

  • TypeError – If a single Buffer is passed instead of a sequence, if a default-stream token (LEGACY_DEFAULT_STREAM / PER_THREAD_DEFAULT_STREAM) is passed, or if the stream is currently in graph capture mode.

Notes

Batching through cuMemcpyBatchAsync requires all three of: cuda.core built against CUDA 13 headers, cuda.bindings 13.0 or newer, and a driver reporting CUDA 13.0 or newer (cuDriverGetVersion() >= 13000). cuda.bindings binds only the CUDA 13.0 revision of the entry point, so a driver that predates it is refused even where it implements the earlier CUDA 12.8 signature.

The driver may execute batch items concurrently and in any order. A batch must therefore not contain copies where the source range of one copy overlaps the destination range of another; such aliasing produces undefined results. Detecting overlaps at runtime is impractical; callers are responsible for ensuring no aliasing exists.

On pre-CUDA 13 installs the copies fall back to a Python-level loop over cuMemcpyAsync, so the potential performance benefit of asynchronous batched copies is not realized. CopyOptions are silently ignored on the fallback path.

On this page