cuda.core.Device

class cuda.core.Device(device_id: Device | int | None = None)

Represent a GPU and act as an entry point for cuda.core features.

This is a singleton object that helps ensure interoperability across multiple libraries imported in the process to both see and use the same GPU device.

While acting as the entry point, many other CUDA resources can be allocated such as streams and buffers. Any Context dependent resource created through this device, will continue to refer to this device’s context.

Newly returned Device objects are thread-local singletons for a specified device.

Note

Will not initialize the GPU.

Parameters:

device_id (int, optional) – Device ordinal to return a Device object for. Default value of None return the currently used device.

Methods

__init__()

allocate(

self,

int size: int,

*,

stream: Stream | GraphBuilder,

) → Buffer

Allocate device memory from a specified stream.

Allocates device memory of size bytes on the specified stream using the memory resource currently associated with this Device.

Note

Device must be initialized.

Parameters:

• size (int) – Number of bytes to allocate.

• stream (Stream | GraphBuilder) – Keyword-only. The stream establishing the stream ordering semantic. Must be passed explicitly; pass self.default_stream to use the default stream.

Returns:

Newly created buffer object.

Return type:

Buffer

can_access_peer(self, peer: Device | int) → bool

Check if this device can access memory from the specified peer device.

Queries whether peer-to-peer memory access is supported between this device and the specified peer device.

Parameters:

peer (Device | int) – The peer device to check accessibility to. Can be a Device object or device ID.

create_context(

self,

options: ContextOptions | None = None,

) → Context

Create a new Context object.

Note

The newly created context will not be set as current.

Parameters:

options (ContextOptions, optional) – Customizable dataclass for context creation options.

Returns:

Newly created context object.

Return type:

Context

create_event(

self,

options: EventOptions | None = None,

) → Event

Create an Event object without recording it to a Stream.

Note

Device must be initialized.

Parameters:

options (EventOptions, optional) – Customizable dataclass for event creation options.

Returns:

Newly created event object.

Return type:

Event

create_graph_builder(self) → GraphBuilder

Create a new GraphBuilder object.

Returns:

Newly created graph builder object.

Return type:

GraphBuilder

create_mipmapped_array(

self,

options: MipmappedArrayOptions,

) → MipmappedArray

Create a MipmappedArray on the current device.

Allocates a mipmapped CUDA array for texture/surface access across levels. The array is created in the current CUDA context, so make this device current with set_current() before calling (mirroring create_stream() / create_event()).

Note

Device must be initialized.

Parameters:

options (MipmappedArrayOptions) – Allocation options (shape, format, channels, levels, surface flag).

Returns:

• MipmappedArray – Newly created mipmapped array.

• .. versionadded:: 1.1.0

create_opaque_array(

self,

options: OpaqueArrayOptions,

) → OpaqueArray

Create an OpaqueArray on the current device.

Allocates an opaque, hardware-laid-out CUDA array for texture/surface access. The array is created in the current CUDA context, so make this device current with set_current() before calling (mirroring create_stream() / create_event()).

Note

Device must be initialized.

Parameters:

options (OpaqueArrayOptions) – Allocation options (shape, format, channels, surface flag).

Returns:

• OpaqueArray – Newly created opaque array.

• .. versionadded:: 1.1.0

create_stream(

self,

obj: IsStreamType | None = None,

options: object = None,

) → Stream

Create a Stream object.

New stream objects can be created in two different ways:

1. Create a new CUDA stream with customizable options.

2. Wrap an existing foreign obj supporting the __cuda_stream__ protocol.

Option (2) internally holds a reference to the foreign object such that the lifetime is managed.

Note

Device must be initialized.

Parameters:

• obj (IsStreamType, optional) – Any object supporting the __cuda_stream__ protocol.

• options (StreamOptions, optional) – Customizable dataclass for stream creation options.

Returns:

Newly created stream object.

Return type:

Stream

create_surface_object(

self,

*,

resource: ResourceDescriptor,

) → SurfaceObject

Create a SurfaceObject on the current device.

Binds an OpaqueArray (via a ResourceDescriptor) as a bindless surface for kernel-side typed load/store. The backing array must have been created with is_surface_load_store=True. The object is created in the current CUDA context, so make this device current with set_current() before calling (mirroring create_stream() / create_event()).

Note

Device must be initialized.

Parameters:

resource (ResourceDescriptor) – Must wrap an OpaqueArray allocated with is_surface_load_store=True.

Returns:

• SurfaceObject – Newly created surface object.

• .. versionadded:: 1.1.0

create_texture_object(

self,

*,

resource: ResourceDescriptor,

options: TextureObjectOptions | None = None,

) → TextureObject

Create a TextureObject on the current device.

Binds a resource (an OpaqueArray / MipmappedArray / linear or pitch2d Buffer, wrapped in a ResourceDescriptor) as a bindless texture for kernel-side sampled reads. The object is created in the current CUDA context, so make this device current with set_current() before calling (mirroring create_stream() / create_event()).

Note

Device must be initialized.

Parameters:

• resource (ResourceDescriptor) – The memory backing the texture.

• options (TextureObjectOptions) – Sampling state (address/filter/read modes, normalization, etc.).

Returns:

• TextureObject – Newly created texture object.

• .. versionadded:: 1.1.0

classmethod get_all_devices(cls) → tuple[Device, ...]

Query the available device instances.

Returns:

A tuple containing instances of available devices.

Return type:

tuple of Device

set_current(

self,

Context ctx: Context | None = None,

) → Context | None

Set device to be used for GPU executions.

Initializes CUDA and sets the calling thread to a valid CUDA context. By default the primary context is used, but optional ctx parameter can be used to explicitly supply a Context object.

Providing a ctx causes the previous set context to be popped and returned.

Parameters:

ctx (Context, optional) – Optional context to push onto this device’s current thread stack.

Returns:

Popped context.

Return type:

Context, optional

Examples

Acts as an entry point of this object. Users always start a code by calling this method, e.g.

>>> from cuda.core import Device
>>> dev0 = Device(0)
>>> dev0.set_current()
>>> # ... do work on device 0 ...

sync(self) → None

Synchronize the device.

Note

Device must be initialized.

to_system_device(self) → 'cuda.core.system.Device'

Get the corresponding cuda.core.system.Device (which is used for NVIDIA Management Library (NVML) access) for this cuda.core.Device (which is used for CUDA access).

The devices are mapped to one another by their UUID.

Returns:

The corresponding system-level device instance used for NVML access.

Return type:

cuda.core.system.Device

Attributes

property arch: str

Return compute capability as a string (e.g., ‘75’ for CC 7.5).

property compute_capability: ComputeCapability

major and minor.

Type:

Return a named tuple with 2 fields

property context: Context

Return the Context associated with this device.

Note

Device must be initialized.

property default_stream: Stream

Return default CUDA Stream associated with this device.

The type of default stream returned depends on if the environment variable CUDA_PYTHON_CUDA_PER_THREAD_DEFAULT_STREAM is set.

If set, returns a per-thread default stream. Otherwise returns the legacy stream.

property device_id: int

Return device ordinal.

property memory_resource: MemoryResource

Return MemoryResource associated with this device.

property name: str

Return the device name.

property pci_bus_id: str

Return a PCI Bus Id string for this device.

property properties: DeviceProperties

Return a DeviceProperties class with information about the device.

property resources: DeviceResources

Return the hardware resource query namespace for this device.

property uuid: str

Return a UUID for the device.

Returns 16-octets identifying the device. If the device is in MIG mode, returns its MIG UUID which uniquely identifies the subscribed MIG compute instance.

Note

MIG UUID is only returned when device is in MIG mode and the driver is older than CUDA 11.4.

The UUID is cached after first access to avoid repeated CUDA API calls.