cuda.core.VirtualMemoryResource#

class cuda.core.VirtualMemoryResource(
device_id: Device | int,
config: VirtualMemoryResourceOptions | None = None,
)#

Create a device memory resource that uses the CUDA VMM APIs to allocate memory.

Parameters:

  • device_id (Device | int) – Device for which a memory resource is constructed.

  • config (VirtualMemoryResourceOptions, optional) – A configuration object for the VirtualMemoryResource

Warning

This is a low-level API that is provided only for convenience. Make sure you fully understand how CUDA Virtual Memory Management works before using this. Other MemoryResource subclasses in cuda.core should already meet the common needs.

Notes

Every buffer this resource returns is a VirtualMemoryBuffer that owns its address reservations, physical allocations and mappings; closing the buffer releases them. deallocate() is not involved in that path.

Methods

__init__(*args, **kwargs)#
allocate(
self,
size_t size,
*,
stream: Stream | GraphBuilder | None = None,
) → VirtualMemoryBuffer#

Allocate a buffer of the given size using CUDA virtual memory.

Parameters:

  • size (int) – The size in bytes of the buffer to allocate. It is rounded up to the allocation granularity; the returned buffer reports the rounded size.

  • stream (Stream | GraphBuilder, optional) – Keyword-only. The allocation itself is synchronous. A real stream is recorded as the buffer’s deallocation stream and synchronized when the buffer closes; with None or a default-stream token the legacy default stream of the resource’s device is recorded instead.

Returns:

A buffer that owns its reservation, physical allocation and mapping.

Return type:

VirtualMemoryBuffer

Raises:

CUDAError – If any CUDA driver API call fails during allocation. Nothing is left allocated when this method raises.

deallocate(
self,
ptr: DevicePointerType,
int size: int,
*,
stream: Stream | GraphBuilder | None = None,
) → None#

Unmap and free one address range that was reserved and mapped outside this resource.

Buffers returned by allocate() and modify_allocation() free themselves when they close and never call this method. It exists for raw pointers wrapped with Buffer.from_handle() with mr set to this resource: the range must be exactly one reservation, and the caller must already have released its own cuMemCreate handle, so the physical memory is freed by the unmap.

Parameters:

  • ptr (DevicePointerType) – The start of the reservation.

  • size (int) – The size of the reservation in bytes.

  • stream (Stream | GraphBuilder, optional) – Keyword-only. If given, stream.sync() is called before the range is unmapped, except for a default-stream token on a host-located resource, which has no context to synchronize in.

modify_allocation(
self,
Buffer buf: Buffer,
size_t new_size,
config: VirtualMemoryResourceOptions | None = None,
) → VirtualMemoryBuffer#

Grow a buffer of this resource to at least new_size bytes.

The buffer passed in stays open and usable. The returned buffer aliases it: both map the same physical memory, which is freed when the last of the two closes. When the driver can extend the address range in place, the returned buffer has the same pointer; otherwise it has a new one and the existing contents are reachable through both.

This method is not thread-safe with respect to two buffers that share an address range.

Parameters:

  • buf (VirtualMemoryBuffer) – A buffer returned by allocate() or by this method.

  • new_size (int) – The requested total size in bytes; rounded up to the granularity.

  • config (VirtualMemoryResourceOptions, optional) – Configuration for the new physical memory chunk only. Existing chunks keep the access they were created with, and the resource’s own configuration is unchanged. It must name the resource’s location_type and passes the same checks as the constructor.

Returns:

buf itself when it already covers new_size; otherwise a new buffer of the rounded size.

Return type:

VirtualMemoryBuffer

Raises:

  • TypeError – If buf did not come from this resource.

  • ValueError – If config names a different location than the resource, or the constructor would reject it.

  • RuntimeError – If buf is closed, or config requests GPUDirect RDMA on a device without support.

  • CUDAError – If a driver call fails. buf is untouched when this method raises.

Attributes

config#

object

Type:

config

device#

object

Type:

device

device_id#

Get the device ID associated with this memory resource.

Returns:

int: CUDA device ID. -1 if the memory resource allocates host memory

is_device_accessible#

Indicates whether the allocated memory is accessible from the device.

is_host_accessible#

Indicates whether the allocated memory is accessible from the host.

is_ipc_enabled#

Return False. Buffers of this resource cannot be shared through IPC descriptors.

is_managed#

bool

Whether buffers allocated by this resource are CUDA managed (unified) memory.

Type:

MemoryResource.is_managed