cuda.core.experimental.DeviceMemoryResource#

class cuda.core.experimental.DeviceMemoryResource(device_id: int | Device, options=None)#

Create a device memory resource managing a stream-ordered memory pool.

Parameters:

  • device_id (int | Device) – Device or Device ordinal for which a memory resource is constructed.

  • options (DeviceMemoryResourceOptions) –

    Memory resource creation options.

    If set to None, the memory resource uses the driver’s current stream-ordered memory pool for the specified device_id. If no memory pool is set as current, the driver’s default memory pool for the device is used.

    If not set to None, a new memory pool is created, which is owned by the memory resource.

    When using an existing (current or default) memory pool, the returned device memory resource does not own the pool (is_handle_owned is False), and closing the resource has no effect.

Notes

To create an IPC-Enabled memory resource (MR) that is capable of sharing allocations between processes, specify ipc_enabled=True in the initializer option. Sharing an allocation is a two-step procedure that involves mapping a memory resource and then mapping buffers owned by that resource. These steps can be accomplished in several ways.

An IPC-enabled memory resource can allocate memory buffers but cannot receive shared buffers. Mapping an MR to another process creates a “mapped memory resource” (MMR). An MMR cannot allocate memory buffers and can only receive shared buffers. MRs and MMRs are both of type DeviceMemoryResource and can be distinguished via DeviceMemoryResource.is_mapped.

An MR is shared via an allocation handle obtained by calling DeviceMemoryResource.get_allocation_handle(). The allocation handle has a platform-specific interpretation; however, memory IPC is currently only supported for Linux, and in that case allocation handles are file descriptors. After sending an allocation handle to another process, it can be used to create an MMR by invoking DeviceMemoryResource.from_allocation_handle().

Buffers can be shared as serializable descriptors obtained by calling Buffer.get_ipc_descriptor(). In a receiving process, a shared buffer is created by invoking Buffer.from_ipc_descriptor() with an MMR and buffer descriptor, where the MMR corresponds to the MR that created the described buffer.

To help manage the association between memory resources and buffers, a registry is provided. Every MR has a unique identifier (UUID). MMRs can be registered by calling DeviceMemoryResource.register() with the UUID of the corresponding MR. Registered MMRs can be looked up via DeviceMemoryResource.from_registry(). When registering MMRs in this way, the use of buffer descriptors can be avoided. Instead, buffer objects can themselves be serialized and transferred directly. Serialization embeds the UUID, which is used to locate the correct MMR during reconstruction.

IPC-enabled memory resources interoperate with the multiprocessing module to provide a simplified interface. This approach can avoid direct use of allocation handles, buffer descriptors, MMRs, and the registry. When using multiprocessing to spawn processes or send objects through communication channels such as multiprocessing.Queue, multiprocessing.Pipe, or multiprocessing.Connection, Buffer objects may be sent directly, and in such cases the process for creating MMRs and mapping buffers will be handled automatically.

For greater efficiency when transferring many buffers, one may also send MRs and buffers separately. When an MR is sent via multiprocessing, an MMR is created and registered in the receiving process. Subsequently, buffers may be serialized and transferred using ordinary pickle methods. The reconstruction procedure uses the registry to find the associated MMR.

Methods

__init__(*args, **kwargs)#
allocate(
self,
size_t size,
stream: Stream = None,
) → Buffer#

Allocate a buffer of the requested size.

Parameters:

  • size (int) – The size of the buffer to allocate, in bytes.

  • stream (Stream, optional) – The stream on which to perform the allocation asynchronously. If None, an internal stream is used.

Returns:

The allocated buffer object, which is accessible on the device that this memory resource was created for.

Return type:

Buffer

close(self)#

Close the device memory resource and destroy the associated memory pool if owned.

deallocate(
self,
ptr: DevicePointerT,
size_t size,
stream: Stream,
)#

Deallocate a buffer previously allocated by this resource.

Parameters:

  • ptr (DevicePointerT) – The pointer or handle to the buffer to deallocate.

  • size (int) – The size of the buffer to deallocate, in bytes.

  • stream (Stream, optional) – The stream on which to perform the deallocation asynchronously. If the buffer is deallocated without an explicit stream, the allocation stream is used.

classmethod from_allocation_handle(
cls,
device_id: int | Device,
alloc_handle: int | IPCAllocationHandle,
) → DeviceMemoryResource#

Create a device memory resource from an allocation handle.

Construct a new DeviceMemoryResource instance that imports a memory pool from a shareable handle. The memory pool is marked as owned, and the resource is associated with the specified device_id.

Parameters:

  • device_id (int | Device) – The ID of the device or a Device object for which the memory resource is created.

  • alloc_handle (int | IPCAllocationHandle) – The shareable handle of the device memory resource to import.

Return type:

A new device memory resource instance with the imported handle.

static from_registry(
uuid: uuid.UUID,
) → DeviceMemoryResource#

Obtain a registered mapped memory resource.

Raises:

RuntimeError – If no mapped memory resource is found in the registry.

get_allocation_handle(
self,
) → IPCAllocationHandle#

Export the memory pool handle to be shared (requires IPC).

The handle can be used to share the memory pool with other processes. The handle is cached in this MemoryResource and owned by it.

Return type:

The shareable handle for the memory pool.

register(
self,
uuid: uuid.UUID,
) → DeviceMemoryResource#

Register a mapped memory resource.

Returns:

  • The registered mapped memory resource. If one was previously registered

  • with the given key, it is returned.

unregister(self)#

Unregister this mapped memory resource.

Attributes