cuda.core.experimental.Device#

class cuda.core.experimental.Device(device_id: int | None = None)#

Represent a GPU and act as an entry point for cuda.core features.

This is a singleton object that helps ensure interoperability across multiple libraries imported in the process to both see and use the same GPU device.

While acting as the entry point, many other CUDA resources can be allocated such as streams and buffers. Any Context dependent resource created through this device, will continue to refer to this device’s context.

Newly returned Device objects are thread-local singletons for a specified device.

Note

Will not initialize the GPU.

Parameters:

device_id (int, optional) – Device ordinal to return a Device object for. Default value of None return the currently used device.

Methods

__init__()#
allocate(
size,
stream: Stream | None = None,
) → Buffer#

Allocate device memory from a specified stream.

Allocates device memory of size bytes on the specified stream using the memory resource currently associated with this Device.

Parameter stream is optional, using a default stream by default.

Note

Device must be initialized.

Parameters:

  • size (int) – Number of bytes to allocate.

  • stream (Stream, optional) – The stream establishing the stream ordering semantic. Default value of None uses default stream.

Returns:

Newly created buffer object.

Return type:

Buffer

create_context(
options: ContextOptions = None,
) → Context#

Create a new Context object.

Note

The newly context will not be set as current.

Parameters:

options (ContextOptions, optional) – Customizable dataclass for context creation options.

Returns:

Newly created context object.

Return type:

Context

create_event(
options: EventOptions | None = None,
) → Event#

Create an Event object without recording it to a Stream.

Note

Device must be initialized.

Parameters:

options (EventOptions, optional) – Customizable dataclass for event creation options.

Returns:

Newly created event object.

Return type:

Event

create_graph_builder() → GraphBuilder#

Create a new GraphBuilder object.

Returns:

Newly created graph builder object.

Return type:

GraphBuilder

create_stream(
obj: IsStreamT | None = None,
options: StreamOptions | None = None,
) → Stream#

Create a Stream object.

New stream objects can be created in two different ways:

  1. Create a new CUDA stream with customizable options.

  2. Wrap an existing foreign obj supporting the __cuda_stream__ protocol.

Option (2) internally holds a reference to the foreign object such that the lifetime is managed.

Note

Device must be initialized.

Parameters:

  • obj (IsStreamT, optional) – Any object supporting the __cuda_stream__ protocol.

  • options (StreamOptions, optional) – Customizable dataclass for stream creation options.

Returns:

Newly created stream object.

Return type:

Stream

set_current(
ctx: Context = None,
) → Context | None#

Set device to be used for GPU executions.

Initializes CUDA and sets the calling thread to a valid CUDA context. By default the primary context is used, but optional ctx parameter can be used to explicitly supply a Context object.

Providing a ctx causes the previous set context to be popped and returned.

Parameters:

ctx (Context, optional) – Optional context to push onto this device’s current thread stack.

Returns:

Popped context.

Return type:

Union[Context, None], optional

Examples

Acts as an entry point of this object. Users always start a code by calling this method, e.g.

>>> from cuda.core.experimental import Device
>>> dev0 = Device(0)
>>> dev0.set_current()
>>> # ... do work on device 0 ...
sync()#

Synchronize the device.

Note

Device must be initialized.

Attributes

property arch: str#

Return compute capability as a string (e.g., ‘75’ for CC 7.5).

property compute_capability: ComputeCapability#

major and minor.

Type:

Return a named tuple with 2 fields

property context: Context#

Return the current Context associated with this device.

Note

Device must be initialized.

property default_stream: Stream#

Return default CUDA Stream associated with this device.

The type of default stream returned depends on if the environment variable CUDA_PYTHON_CUDA_PER_THREAD_DEFAULT_STREAM is set.

If set, returns a per-thread default stream. Otherwise returns the legacy stream.

property device_id: int#

Return device ordinal.

property memory_resource: MemoryResource#

Return MemoryResource associated with this device.

property name: str#

Return the device name.

property pci_bus_id: str#

Return a PCI Bus Id string for this device.

property properties: DeviceProperties#

Return a DeviceProperties class with information about the device.

property uuid: str#

Return a UUID for the device.

Returns 16-octets identifying the device. If the device is in MIG mode, returns its MIG UUID which uniquely identifies the subscribed MIG compute instance.

Note

MIG UUID is only returned when device is in MIG mode and the driver is older than CUDA 11.4.