cuda.core.VirtualMemoryBuffer#

class cuda.core.VirtualMemoryBuffer#

A Buffer returned by VirtualMemoryResource.

The buffer owns its address reservations, physical allocations and mappings through its device pointer handle; closing it is the only way to release them. A buffer returned by VirtualMemoryResource.modify_allocation() aliases the buffer it was grown from: the two share their physical memory, and that memory is freed when the last buffer that maps it closes.

Methods

__init__(*args, **kwargs)#
close(
self,
stream: Stream | GraphBuilder | None = None,
) → None#

Release this buffer’s share of its address range.

The mappings, reservations and physical allocations go away when the last buffer that maps them closes. Before it unmaps, the resource synchronizes every deallocation stream the buffers of the range recorded. Virtual memory deallocation is synchronous and cannot be captured. When the stream this close uses, given or recorded, is not a default stream and is capturing, the call raises and leaves the buffer open. A default stream is checked when the range is released instead: if synchronizing it would disturb a capture in its context, the release reports a CUDAWarning and unmaps without synchronizing that stream.

Parameters:

stream (Stream | GraphBuilder, optional) – If given, replaces the recorded deallocation stream, as for Buffer.close().

copy_from(
self,
Buffer src: Buffer,
*,
stream: Stream | GraphBuilder,
options: CopyOptions | None = None,
) → None#

Copy from the src buffer to this buffer asynchronously on the given stream.

Parameters:

  • src (Buffer) – Source buffer to copy data from

  • stream (Stream | GraphBuilder) – Keyword argument specifying the stream for the asynchronous copy

  • options (CopyOptions, optional) – Transfer hints (source access order, location hints, overlap mode). Honored when cuda.bindings and the driver are both CUDA 13.2 or newer. Not accepted with LEGACY_DEFAULT_STREAM; use PER_THREAD_DEFAULT_STREAM instead. Not accepted with a capturing stream either, since a graph cannot represent these attributes; use graph.GraphNode.memcpy() for a plain, non-attributed copy node, or pass options=None. On an older cuda.bindings/driver, src_access_order values of STREAM and ANY are silently ignored; DURING_API_CALL raises instead of silently downgrading its guarantee.

Raises:

  • TypeError – If options is not a CopyOptions instance, or if options is given together with LEGACY_DEFAULT_STREAM or a stream currently in graph capture mode.

  • RuntimeError – If options.src_access_order is DURING_API_CALL and cuda.bindings or the driver is older than CUDA 13.2: falling back to a plain copy cannot honor that guarantee.

copy_to(
self,
Buffer dst: Buffer | None = None,
*,
stream: Stream | GraphBuilder,
options: CopyOptions | None = None,
) → Buffer#

Copy from this buffer to the dst buffer asynchronously on the given stream.

Copies the data from this buffer to the provided dst buffer. If the dst buffer is not provided, then a new buffer is first allocated using the associated memory resource before the copy.

Parameters:

  • dst (Buffer, optional) – Destination buffer to copy data to. If not provided, a new buffer is allocated using this buffer’s memory resource.

  • stream (Stream | GraphBuilder) – Keyword argument specifying the stream for the asynchronous copy

  • options (CopyOptions, optional) – Transfer hints (source access order, location hints, overlap mode). Honored when cuda.bindings and the driver are both CUDA 13.2 or newer. Not accepted with LEGACY_DEFAULT_STREAM; use PER_THREAD_DEFAULT_STREAM instead. Not accepted with a capturing stream either, since a graph cannot represent these attributes; use graph.GraphNode.memcpy() for a plain, non-attributed copy node, or pass options=None. On an older cuda.bindings/driver, src_access_order values of STREAM and ANY are silently ignored; DURING_API_CALL raises instead of silently downgrading its guarantee.

Raises:

  • TypeError – If options is not a CopyOptions instance, or if options is given together with LEGACY_DEFAULT_STREAM or a stream currently in graph capture mode.

  • RuntimeError – If options.src_access_order is DURING_API_CALL and cuda.bindings or the driver is older than CUDA 13.2: falling back to a plain copy cannot honor that guarantee.

fill(
self,
value: int | BufferProtocol,
*,
stream: Stream | GraphBuilder,
) → None#

Fill this buffer with a repeating byte pattern.

Parameters:
Raises:

  • TypeError – If value is not an int and does not support the buffer protocol.

  • ValueError – If value byte length is not 1, 2, or 4. If buffer size is not divisible by value byte length.

  • OverflowError – If int value is outside [0, 256).

static from_handle(
ptr: DevicePointerType,
size_t size,
MemoryResource mr: MemoryResource | None = None,
owner: object | None = None,
*,
stream: Stream | GraphBuilder | None = None,
) → Buffer#

Create a new Buffer object from a pointer.

Parameters:

  • ptr (DevicePointerType) – Allocated buffer handle object

  • size (int) – Memory size of the buffer

  • mr (MemoryResource, optional) – Memory resource associated with the buffer. When provided, MemoryResource.deallocate() is called when the buffer is closed or garbage collected.

  • owner (object, optional) – An object holding external allocation that the ptr points to. The reference is kept as long as the buffer is alive. The owner and mr cannot be specified together.

  • stream (Stream | GraphBuilder, optional) – Keyword-only. The stream used to order the buffer’s deallocation when mr owns the pointer. Defaults to default_stream(). Recording a default-stream token requires a CUDA context to be current. Host-only resources (mr.is_device_accessible is False) record no stream and need no context. If the buffer may be freed from a different host thread, pass a stream other than the per-thread default stream, which refers to a different stream on each thread.

Note

When neither mr nor owner is specified, this creates a non-owning reference. The pointer will NOT be freed when the Buffer is closed or garbage collected.

classmethod from_ipc_descriptor(
cls,
mr: DeviceMemoryResource | PinnedMemoryResource,
IPCBufferDescriptor ipc_descriptor: IPCBufferDescriptor,
*,
Stream stream: Stream,
) → Buffer#

Import a buffer that was exported from another process.

Parameters:

  • mr (DeviceMemoryResource | PinnedMemoryResource) – The IPC-enabled memory resource matching the exporting process.

  • ipc_descriptor (IPCBufferDescriptor) – The descriptor exported from another process.

  • stream (Stream) – Keyword-only. The stream used for asynchronous deallocation when the buffer is closed or garbage collected.

Note

The descriptor payload and size are supplied by the exporting peer and must be treated as untrusted input unless the peer is known to be cooperating.

set_deallocation_stream(
self,
stream: Stream | GraphBuilder,
) → None#

Change the stream that orders this buffer’s eventual deallocation.

The buffer remains open and usable. A later close() without a stream, garbage collection, or release of the final retained device pointer handle uses the replacement stream.

This method does not synchronize streams or establish dependencies. The caller must ensure that allocation and all accesses are ordered before the deallocation on stream.

Parameters:

stream (Stream | GraphBuilder) – The stream to use for eventual asynchronous deallocation.

Raises:

  • RuntimeError – If the buffer is already closed, or if a default-stream token cannot be bound because no CUDA context is current.

  • TypeError – If stream is None or is not an accepted stream object.

Notes

Synchronizing concurrent mutation and destruction of the same buffer is the caller’s responsibility.

Attributes

device_id#

int

Return the device ordinal of this buffer, or -1 for memory not bound to a device.

Type:

Buffer.device_id

handle#

int

Return the buffer handle object.

Caution

This handle is a Python object. To get the memory address of the underlying C handle, call int(Buffer.handle).

Type:

Buffer.handle

ipc_descriptor#

IPCBufferDescriptor

Descriptor for sharing this buffer with other processes.

Type:

Buffer.ipc_descriptor

is_closed#

bool

Whether this buffer has been closed.

Type:

Buffer.is_closed

is_device_accessible#

bool

Return True if this buffer can be accessed by the GPU, otherwise False.

Type:

Buffer.is_device_accessible

is_host_accessible#

bool

Return True if this buffer can be accessed by the CPU, otherwise False.

Type:

Buffer.is_host_accessible

is_managed#

bool

Return True if this buffer is CUDA managed (unified) memory, otherwise False.

Type:

Buffer.is_managed

is_mapped#

bool

Return True if this buffer is mapped into the process via IPC.

Type:

Buffer.is_mapped

memory_resource#

MemoryResource

Return the memory resource associated with this buffer.

Type:

Buffer.memory_resource

owner#

object

Return the object holding external allocation.

Type:

Buffer.owner

size#

int

Return the memory size of this buffer.

Type:

Buffer.size