cuda.bindings._v2.driver.memcpy_batch_async#

cuda.bindings._v2.driver.memcpy_batch_async(
dsts,
srcs,
sizes,
attrs,
attrs_idxs,
intptr_t h_stream,
)#

Perform a batch of memory copies asynchronously.

The batch as a whole executes in stream order but copies within a batch are not guaranteed to execute in any specific order. This API only supports pointer-to-pointer copies; for copies involving CUDA arrays use cuMemcpy3DBatchAsync.

Every copy in the batch must be associated with a set of attributes via attrs. Each entry in attrs can apply to more than one copy; the attrs_idxs array gives the index of the first copy each attribute entry applies to. The first entry in attrs_idxs must always be 0, entries must be strictly increasing, and len(attrs) must be ≤ len(dsts).

CUmemcpyAttributes.srcAccessOrder controls source access ordering:

  • CU_MEMCPY_SRC_ACCESS_ORDER_STREAM: source is accessed in stream order.

  • CU_MEMCPY_SRC_ACCESS_ORDER_DURING_API_CALL: source access is out of stream order but all accesses complete before the API returns (suited for ephemeral stack variables; allows the driver to optimize the copy and removes the need for a stream sync after the call).

  • CU_MEMCPY_SRC_ACCESS_ORDER_ANY: source access is out of stream order and accesses may occur even after the API returns (suited for non-CUDA-allocated host pointers such as those from malloc).

Each copy must have a valid CUmemcpyAttributes with an appropriate srcAccessOrder; otherwise CUDA_ERROR_INVALID_VALUE is returned.

dsts, srcs, and sizes must all have the same length; count is inferred from that length. attrs and attrs_idxs must both be None or both be provided with the same length; numAttrs is inferred from that length.

Parameters:

  • dsts – Buffer of destination device pointers (e.g. numpy uint64 array).

  • srcs – Buffer of source device pointers (e.g. numpy uint64 array).

  • sizes – Buffer of copy sizes in bytes (e.g. numpy uint64 array).

  • attrs – Buffer of MemCpyAttributes_v1 values, or None.

  • attrs_idxs – Buffer of per-attribute copy indices parallel to attrs, or None if attrs is None.

  • h_stream (intptr_t) – Stream on which to perform the copies.

See also

cuMemcpyBatchAsync

On this page