cuda.bindings._v2.driver.mem_prefetch_batch_async#

cuda.bindings._v2.driver.mem_prefetch_batch_async( intptr_t dptrs, intptr_t sizes, size_t count, prefetch_locs, intptr_t prefetch_loc_idxs, size_t num_prefetch_locs, unsigned long long flags, intptr_t h_stream, )#

Performs a batch of memory prefetches asynchronously.

Performs a batch of memory prefetches. The batch as a whole executes in stream order but operations within a batch are not guaranteed to execute in any specific order. All devices in the system must have a non-zero value for the device attribute CU_DEVICE_ATTRIBUTE_CONCURRENT_MANAGED_ACCESS otherwise the API will return an error. The semantics of the individual prefetch operations are as described in cuMemPrefetchAsync. Performs memory prefetch on address ranges specified in dptrs and sizes. Both arrays must be of the same length as specified by count. Each memory range specified must refer to managed memory allocated via cuMemAllocManaged or declared via managed variables or it may also refer to system-allocated memory when all devices have a non-zero value for CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS. The prefetch location for every operation in the batch is specified in the prefetch_locs array. Each entry in this array can apply to more than one operation. This can be done by specifying in the prefetch_loc_idxs array, the index of the first prefetch operation that the corresponding entry in the prefetch_locs array applies to. Both prefetch_locs and prefetch_loc_idxs must be of the same length as specified by num_prefetch_locs. For example, if a batch has 10 prefetches listed in dptrs/sizes, the first 4 of which are to be prefetched to one location and the remaining 6 are to be prefetched to another, then num_prefetch_locs will be 2, prefetch_loc_idxs will be {0, 4} and prefetch_locs will contain the two locations. Note the first entry in prefetch_loc_idxs must always be 0. Also, each entry must be greater than the previous entry and the last entry should be less than count. Furthermore, num_prefetch_locs must be lesser than or equal to count.

Parameters:

dptrs (intptr_t) – Array of pointers to be prefetched.
sizes (intptr_t) – Array of sizes for memory prefetch operations.
count (size_t) – Size of dptrs and sizes arrays.
prefetch_locs (intptr_t) – Array of locations to prefetch to.
prefetch_loc_idxs (intptr_t) – Array of indices to specify which operands each entry in the prefetch_locs array applies to. The locations specified in prefetch_locs[k] will be applied to copies starting from prefetch_loc_idxs[k] through prefetch_loc_idxs[k+1] - 1. Also prefetch_locs[num_prefetch_locs - 1] will apply to prefetches starting from prefetch_loc_idxs[num_prefetch_locs - 1] through count - 1.
num_prefetch_locs (size_t) – Size of prefetch_locs and prefetch_loc_idxs arrays.
flags (unsigned long long) – Flags reserved for future use. Must be zero.
h_stream (intptr_t) – The stream to enqueue the operations in. Must not be legacy NULL stream.