driver_v2

See also

See the migration guide for the differences between this module and the legacy driver module.

Data types used by CUDA driver

Uuid	Empty-initialize an instance of CUuuid.
MemFabricHandle_v1	Empty-initialize an instance of CUmemFabricHandle_v1.
IpcEventHandle_v1	Empty-initialize an instance of CUipcEventHandle_v1.
IpcMemHandle_v1	Empty-initialize an instance of CUipcMemHandle_v1.
StreamBatchMemOpParams_v1	Empty-initialize an array of CUstreamBatchMemOpParams_v1.
BatchMemOpNodeParams_v1	Empty-initialize an instance of CUDA_BATCH_MEM_OP_NODE_PARAMS_v1.
BatchMemOpNodeParams_v2	Empty-initialize an instance of CUDA_BATCH_MEM_OP_NODE_PARAMS_v2.
AsyncNotificationInfo	Empty-initialize an instance of CUasyncNotificationInfo.
AccessPolicyWindow_v1	Empty-initialize an instance of CUaccessPolicyWindow_v1.
KernelNodeParams_v2	Empty-initialize an instance of CUDA_KERNEL_NODE_PARAMS_v2.
KernelNodeParams_v3	Empty-initialize an instance of CUDA_KERNEL_NODE_PARAMS_v3.
MemSetNodeParams_v1	Empty-initialize an instance of CUDA_MEMSET_NODE_PARAMS_v1.
MemSetNodeParams_v2	Empty-initialize an instance of CUDA_MEMSET_NODE_PARAMS_v2.
HostNodeParams_v1	Empty-initialize an instance of CUDA_HOST_NODE_PARAMS_v1.
HostNodeParams_v2	Empty-initialize an instance of CUDA_HOST_NODE_PARAMS_v2.
ConditionalNodeParams	Empty-initialize an instance of CUDA_CONDITIONAL_NODE_PARAMS.
GraphEdgeData	Empty-initialize an array of CUgraphEdgeData.
GraphInstantiateParams	Empty-initialize an instance of CUDA_GRAPH_INSTANTIATE_PARAMS.
LaunchMemSyncDomainMap	Empty-initialize an instance of CUlaunchMemSyncDomainMap.
LaunchAttributeValue	Empty-initialize an instance of CUlaunchAttributeValue.
LaunchAttribute	Empty-initialize an instance of CUlaunchAttribute.
LaunchConfig	Empty-initialize an instance of CUlaunchConfig.
ExecAffinitySmCount_v1	Empty-initialize an instance of CUexecAffinitySmCount_v1.
ExecAffinityParam_v1	Empty-initialize an array of CUexecAffinityParam_v1.
CtxCigParam	Empty-initialize an instance of CUctxCigParam.
CtxCreateParams	Empty-initialize an instance of CUctxCreateParams.
StreamCigParam	Empty-initialize an instance of CUstreamCigParam.
StreamCigCaptureParams	Empty-initialize an instance of CUstreamCigCaptureParams.
LibraryHostUniversalFunctionAndDataTable	Empty-initialize an instance of CUlibraryHostUniversalFunctionAndDataTable.
MemCpy2d_v2	Empty-initialize an instance of CUDA_MEMCPY2D_v2.
MemCpy3d_v2	Empty-initialize an instance of CUDA_MEMCPY3D_v2.
MemCpy3dPeer_v1	Empty-initialize an instance of CUDA_MEMCPY3D_PEER_v1.
MemCpyNodeParams	Empty-initialize an instance of CUDA_MEMCPY_NODE_PARAMS.
ArrayDescriptor_v2	Empty-initialize an instance of CUDA_ARRAY_DESCRIPTOR_v2.
Array3dDescriptor_v2	Empty-initialize an instance of CUDA_ARRAY3D_DESCRIPTOR_v2.
ArraySparseProperties_v1	Empty-initialize an instance of CUDA_ARRAY_SPARSE_PROPERTIES_v1.
ArrayMemoryRequirements_v1	Empty-initialize an instance of CUDA_ARRAY_MEMORY_REQUIREMENTS_v1.
ResourceDesc_v1	Empty-initialize an instance of CUDA_RESOURCE_DESC_v1.
TextureDesc_v1	Empty-initialize an instance of CUDA_TEXTURE_DESC_v1.
ResourceViewDesc_v1	Empty-initialize an instance of CUDA_RESOURCE_VIEW_DESC_v1.
TensorMap	Empty-initialize an instance of CUtensorMap.
PointerAttributeP2PTokens_v1	Empty-initialize an instance of CUDA_POINTER_ATTRIBUTE_P2P_TOKENS_v1.
ExternalMemoryHandleDesc_v1	Empty-initialize an instance of CUDA_EXTERNAL_MEMORY_HANDLE_DESC_v1.
ExternalMemoryBufferDesc_v1	Empty-initialize an instance of CUDA_EXTERNAL_MEMORY_BUFFER_DESC_v1.
ExternalMemoryMipmappedArrayDesc_v1	Empty-initialize an instance of CUDA_EXTERNAL_MEMORY_MIPMAPPED_ARRAY_DESC_v1.
ExternalSemaphoreHandleDesc_v1	Empty-initialize an instance of CUDA_EXTERNAL_SEMAPHORE_HANDLE_DESC_v1.
ExternalSemaphoreSignalParams_v1	Empty-initialize an instance of CUDA_EXTERNAL_SEMAPHORE_SIGNAL_PARAMS_v1.
ExternalSemaphoreWaitParams_v1	Empty-initialize an instance of CUDA_EXTERNAL_SEMAPHORE_WAIT_PARAMS_v1.
ExtSemSignalNodeParams_v1	Empty-initialize an instance of CUDA_EXT_SEM_SIGNAL_NODE_PARAMS_v1.
ExtSemSignalNodeParams_v2	Empty-initialize an instance of CUDA_EXT_SEM_SIGNAL_NODE_PARAMS_v2.
ExtSemWaitNodeParams_v1	Empty-initialize an instance of CUDA_EXT_SEM_WAIT_NODE_PARAMS_v1.
ExtSemWaitNodeParams_v2	Empty-initialize an instance of CUDA_EXT_SEM_WAIT_NODE_PARAMS_v2.
ArrayMapInfo_v1	Empty-initialize an array of CUarrayMapInfo_v1.
MemLocation_v1	Empty-initialize an instance of CUmemLocation_v1.
MemAllocationProp_v1	Empty-initialize an instance of CUmemAllocationProp_v1.
MulticastObjectProp_v1	Empty-initialize an instance of CUmulticastObjectProp_v1.
MemAccessDesc_v1	Empty-initialize an array of CUmemAccessDesc_v1.
GraphExecUpdateResultInfo_v1	Empty-initialize an instance of CUgraphExecUpdateResultInfo_v1.
MemPoolProps_v1	Empty-initialize an instance of CUmemPoolProps_v1.
MemPoolPtrExportData_v1	Empty-initialize an instance of CUmemPoolPtrExportData_v1.
MemCpyAttributes_v1	Empty-initialize an instance of CUmemcpyAttributes_v1.
Offset3D_v1	Empty-initialize an instance of CUoffset3D_v1.
Extent3D_v1	Empty-initialize an instance of CUextent3D_v1.
MemCpy3dOperand_v1	Empty-initialize an instance of CUmemcpy3DOperand_v1.
MemCpy3dBatchOp_v1	Empty-initialize an instance of CUDA_MEMCPY3D_BATCH_OP_v1.
MemAllocNodeParams_v1	Empty-initialize an instance of CUDA_MEM_ALLOC_NODE_PARAMS_v1.
MemAllocNodeParams_v2	Empty-initialize an instance of CUDA_MEM_ALLOC_NODE_PARAMS_v2.
MemFreeNodeParams	Empty-initialize an instance of CUDA_MEM_FREE_NODE_PARAMS.
ChildGraphNodeParams	Empty-initialize an instance of CUDA_CHILD_GRAPH_NODE_PARAMS.
EventRecordNodeParams	Empty-initialize an instance of CUDA_EVENT_RECORD_NODE_PARAMS.
EventWaitNodeParams	Empty-initialize an instance of CUDA_EVENT_WAIT_NODE_PARAMS.
GraphNodeParams	Empty-initialize an instance of CUgraphNodeParams.
CheckpointLockArgs	Empty-initialize an instance of CUcheckpointLockArgs.
CheckpointCheckpointArgs	Empty-initialize an instance of CUcheckpointCheckpointArgs.
CheckpointGpuPair	Empty-initialize an instance of CUcheckpointGpuPair.
CheckpointRestoreArgs	Empty-initialize an instance of CUcheckpointRestoreArgs.
CheckpointUnlockArgs	Empty-initialize an instance of CUcheckpointUnlockArgs.
IpcMemFlags	CUDA Ipc Mem Flags
MemAttachFlags	CUDA Mem Attach Flags
CtxFlags	Context creation flags
EventSchedFlags	Event sched flags
ContextFlags	NVCL context scheduling flags
HostTaskSyncMode	See CUhostTaskSyncMode.
StreamFlags	Stream creation flags
EventFlags	Event creation flags
EventRecordFlags	Event record flags
EventWaitFlags	Event wait flags
AtomicOperation	CUDA-valid Atomic Operations
AtomicOperationCapability	CUDA-valid Atomic Operation capabilities
StreamWaitValueFlags	Flags for cuStreamWaitValue32 and cuStreamWaitValue64
StreamWriteValueFlags	Flags for cuStreamWriteValue32
StreamBatchMemOpType	Operations for cuStreamBatchMemOp
StreamMemoryBarrierFlags	Flags for CUstreamBatchMemOpParams.memoryBarrier
StreamAtomicReductionOpType	Atomic reduction operation types for CUstreamBatchMemOpParams::atomicReduction::reductionOp
StreamAtomicReductionDataType	Atomic reduction data types for CUstreamBatchMemOpParams::atomicReduction::dataType
OccupancyFlags	Occupancy calculator flag
StreamUpdateCaptureDependenciesFlags	Flags for cuStreamUpdateCaptureDependencies
AsyncNotificationType	Types of async notification that can be sent
ArrayFormat	Array formats
AddressMode	Texture reference addressing modes
FilterMode	Texture reference filtering modes
DeviceAttribute	Device properties
PointerAttribute	Pointer information
FunctionAttribute	Function properties
FuncCache	Function cache configurations
SharedCarveout	Shared memory carveout configurations.
MemoryType	Memory types
ComputeMode	Compute Modes
MemAdvise	Memory advise values
MemRangeAttribute	See CUmem_range_attribute.
JitOption	Online compiler and linker options
JitTarget	Online compilation targets
JitFallback	Cubin matching fallback strategies
JitCacheMode	Caching modes for dlcm
JitInputType	Device code formats
GraphicsRegisterFlags	Flags to register a graphics resource
GraphicsMapResourceFlags	Flags for mapping and unmapping interop resources
ArrayCubemapFace	Array indices for cube faces
Limit	Limits
ResourceType	Resource types
AccessProperty	Specifies performance hint with CUaccessPolicyWindow for hitProp and missProp members.
GraphConditionalNodeType	Conditional node types
GraphNodeType	Graph node types
GraphDependencyType	Type annotations that can be applied to graph edges as part of CUgraphEdgeData.
GraphInstantiateResult	Graph instantiation results
SynchronizationPolicy	See CUsynchronizationPolicy.
ClusterSchedulingPolicy	Cluster scheduling policies.
LaunchMemSyncDomain	Memory Synchronization Domain A kernel can be launched in a specified memory synchronization domain that affects all memory operations issued by that kernel.
LaunchAttributePortableClusterMode	Enum for defining applicability of portable cluster size, used with cuLaunchKernelEx
SharedMemoryMode	Shared memory related attributes for use with cuLaunchKernelEx
LaunchAttributeID	Launch attributes enum; used as id field of CUlaunchAttribute
StreamCaptureStatus	Possible stream capture statuses returned by cuStreamIsCapturing
StreamCaptureMode	Possible modes for stream capture thread interactions.
DriverProcAddressFlags	Flags to specify search options.
DriverProcAddressQueryResult	Flags to indicate search status.
ExecAffinityType	Execution Affinity Types
CigDataType	See CUcigDataType.
StreamCigDataType	See CUstreamCigDataType.
LibraryOption	Library options to be specified with cuLibraryLoadData() or cuLibraryLoadFromFile()
Result	Error codes
DeviceP2PAttribute	P2P Attributes
ResourceViewFormat	Resource view format
TensorMapDataType	Tensor map data type
TensorMapInterleave	Tensor map interleave layout type
TensorMapSwizzle	Tensor map swizzling mode of shared memory banks
TensorMapL2Promotion	Tensor map L2 promotion type
TensorMapFloatOOBFill	Tensor map out-of-bounds fill type
TensorMapIm2ColWideMode	Tensor map Im2Col wide mode
PointerAttributeAccessFlags	Access flags that specify the level of access the current context's device has on the memory referenced.
ExternalMemoryHandleType	External memory handle types
ExternalSemaphoreHandleType	External semaphore handle types
MemAllocationHandleType	Flags for specifying particular handle types
MemAccessFlags	Specifies the memory protection flags for mapping.
MemLocationType	Specifies the type of location
MemAllocationType	Defines the allocation types available
MemAllocationGranularityFlags	Flag for requesting different optimal and required granularities for an allocation.
MemRangeHandleType	Specifies the handle type for address range
MemRangeFlags	Flag for requesting handle type for address range.
ArraySparseSubresourceType	Sparse subresource types
MemOperationType	Memory operation types
MemHandleType	Memory handle types
MemAllocationCompType	Specifies compression attribute for an allocation.
MulticastGranularityFlags	Flags for querying different granularities for a multicast object
GraphExecUpdateResult	CUDA Graph Update error types
MemPoolAttribute	CUDA memory pool attributes
MemCpyFlags	Flags to specify for copies within a batch.
MemCpySrcAccessOrder	These flags allow applications to convey the source access ordering CUDA must maintain.
MemCpy3DOperandType	These flags allow applications to convey the operand type for individual copies specified in cuMemcpy3DBatchAsync.
GraphMemAttribute	See CUgraphMem_attribute.
GraphChildGraphNodeOwnership	Child graph node ownership
FlushGPUDirectRDMAWritesOptions	Bitmasks for CU_DEVICE_ATTRIBUTE_GPU_DIRECT_RDMA_FLUSH_WRITES_OPTIONS
GPUDirectRDMAWritesOrdering	Platform native ordering for GPUDirect RDMA writes
FlushGPUDirectRDMAWritesScope	The scopes for cuFlushGPUDirectRDMAWrites
FlushGPUDirectRDMAWritesTarget	The targets for cuFlushGPUDirectRDMAWrites
GraphDebugDotFlags	The additional write options for cuGraphDebugDotPrint
UserObjectFlags	Flags for user objects for graphs
UserObjectRetainFlags	Flags for retaining user object references for graphs
GraphInstantiateFlags	Flags for instantiating a graph
DeviceNumaConfig	CUDA device NUMA configuration
ProcessState	CUDA Process States
EglFrameType	CUDA EglFrame type - array or pointer
EglResourceLocationFlags	Resource location flags- sysmem or vidmem For CUDA context on iGPU, since video and system memory are equivalent - these flags will not have an effect on the execution.
EglColorFormat	CUDA EGL Color Format - The different planar and multiplanar formats currently supported for CUDA_EGL interops.
Array3dFlags	Flags for Array3dDescriptor_v2.flags_ (CUDA_ARRAY3D_*).
ArraySparsePropertiesFlags	Flags in ArraySparseProperties_v1.flags_ (CU_ARRAY_SPARSE_PROPERTIES_*).
ComputeTargetBase	Offsets added to a compute capability to form a JitTarget with accelerated or family features (CU_COMPUTE_*_TARGET_BASE).
ExternalMemoryFlags	Flags for ExternalMemoryHandleDesc_v1.flags_ (CUDA_EXTERNAL_MEMORY_*).
ExternalSemaphoreSignalFlags	Flags for ExternalSemaphoreSignalParams_v1.flags_ (CUDA_EXTERNAL_SEMAPHORE_SIGNAL_*).
ExternalSemaphoreWaitFlags	Flags for ExternalSemaphoreWaitParams_v1.flags_ (CUDA_EXTERNAL_SEMAPHORE_WAIT_*).
GLMapFlags	Flags to map or unmap a resource
GraphConditionalHandleFlags	Flags for graph_conditional_handle_create (CU_GRAPH_COND_*).
GraphKernelNodePort	Outgoing ports of a kernel node, for GraphEdgeData.from_port (CU_GRAPH_KERNEL_NODE_PORT_*).
MemCreateUsage	Usage hints for MemAllocationProp_v1.alloc_flags (CU_MEM_CREATE_USAGE_*).
MemHostAllocFlags	Flags for mem_host_alloc (CU_MEMHOSTALLOC_*).
MemHostRegisterFlags	Flags for mem_host_register_v2 (CU_MEMHOSTREGISTER_*).
MemPoolCreateUsage	Usage hints for MemPoolProps_v1.usage (CU_MEM_POOL_CREATE_USAGE_*).
NvSciSyncAttr	NvSciSync attribute directions (CUDA_NVSCISYNC_ATTR_*).
OutputMode	Profiler Output Modes
SpecialDevice	Special int device ordinals (CU_DEVICE_*).
SpecialStream	Special stream handles, as int (CU_STREAM_*).
DevSmResourceGroupParams	Empty-initialize an array of CU_DEV_SM_RESOURCE_GROUP_PARAMS.
StreamMemOpAtomicReductionParams	Empty-initialize an instance of CUstreamMemOpAtomicReductionParams_st.
StreamMemOpFlushRemoteWritesParams	Empty-initialize an instance of CUstreamMemOpFlushRemoteWritesParams_st.
StreamMemOpMemoryBarrierParams	Empty-initialize an instance of CUstreamMemOpMemoryBarrierParams_st.
StreamMemOpWaitValueParams	Empty-initialize an instance of CUstreamMemOpWaitValueParams_st.
StreamMemOpWriteValueParams	Empty-initialize an instance of CUstreamMemOpWriteValueParams_st.
DevProp_v1	Empty-initialize an instance of CUdevprop_v1.
LaunchParams_v1	Empty-initialize an instance of CUDA_LAUNCH_PARAMS_v1.
CooperativeLaunchMultiDeviceFlags	Flags for launch_cooperative_kernel_multi_device (CUDA_COOPERATIVE_LAUNCH_MULTI_DEVICE_*).
LaunchParam	Markers in the extra array of launch_kernel (CU_LAUNCH_PARAM_*).
NvclEventFlags	NVCL event scheduling flags
ParamTexRef	Special texture reference index for param_set_tex_ref (CU_PARAM_TR_*).
SharedConfig	[Deprecated] Shared memory configurations
TexRefArrayFlags	Flags for tex_ref_set_array (CU_TRSA_*).
TexRefFlags	Flags for tex_ref_set_flags (CU_TRSF_*).

Error Handling

This section describes the error handling functions of the low-level CUDA driver application programming interface.

AlreadyAcquiredError
AlreadyMappedError
ArrayIsMappedError
AssertError
CallRequiresNewerDriverError
CapturedEventError
CdpNotSupportedError
CdpVersionMismatchError
CompatNotSupportedOnDeviceError
ContainedError
ContextAlreadyCurrentError
ContextAlreadyInUseError
ContextIsDestroyedError
CooperativeLaunchTooLargeError
DeinitializedError
DeviceNotLicensedError
DeviceUnavailableError
DriverError
EccUncorrectableError
ExternalDeviceError
FabricNotReadyError
DriverFileNotFoundError
FunctionNotLoadedError
GraphExecUpdateFailureError
GraphRecaptureFailureError
HardwareStackError
HostMemoryAlreadyRegisteredError
HostMemoryNotRegisteredError
IllegalAddressError
IllegalInstructionError
IllegalStateError
InsufficientLoaderVersionError
InvalidAddressSpaceError
InvalidClusterSizeError
InvalidContextError
InvalidDeviceError
InvalidGraphicsContextError
InvalidHandleError
InvalidImageError
InvalidPcError
InvalidPtxError
InvalidResourceConfigurationError
InvalidResourceTypeError
InvalidSourceError
InvalidValueError
JitCompilationDisabledError
JitCompilerNotFoundError
KeyRotationError
LaunchFailedError
LaunchIncompatibleTexturingError
LaunchOutOfResourcesError
LaunchTimeoutError
LossyQueryError
MapFailedError
MisalignedAddressError
MpsClientTerminatedError
MpsConnectionFailedError
MpsMaxClientsReachedError
MpsMaxConnectionsReachedError
MpsRpcFailureError
MpsServerNotReadyError
MulticastResourceFullError
NoBinaryForGpuError
NoDeviceError
NotFoundError
NotInitializedError
NotMappedAsArrayError
NotMappedAsPointerError
NotMappedError
NotPermittedError
NotReadyError
NotSupportedError
NvlinkUncorrectableError
OperatingSystemError
OutOfMemoryError
PeerAccessAlreadyEnabledError
PeerAccessNotEnabledError
PeerAccessUnsupportedError
PrimaryContextActiveError
ProfilerAlreadyStartedError
ProfilerAlreadyStoppedError
ProfilerDisabledError
ProfilerNotInitializedError
SharedObjectInitFailedError
SharedObjectSymbolNotFoundError
StreamCaptureImplicitError
StreamCaptureInvalidatedError
StreamCaptureIsolationError
StreamCaptureMergeError
StreamCaptureUnjoinedError
StreamCaptureUnmatchedError
StreamCaptureUnsupportedError
StreamCaptureWrongThreadError
StreamDetachedError
StubLibraryError
SystemDriverMismatchError
SystemNotReadyError
TensorMemoryLeakError
DriverTimeoutError
TooManyPeersError
UnknownError
UnmapFailedError
UnsupportedDevsideSyncError
UnsupportedExecAffinityError
UnsupportedLimitError
UnsupportedPtxVersionError

get_error_string	Gets the string description of an error code.
get_error_name	Gets the string representation of an error code enum name.

Initialization

This section describes the initialization functions of the low-level CUDA driver application programming interface.

init	Initialize the CUDA driver API Initializes the driver API and must be called before any other function from the driver API in the current process.

Version Management

This section describes the version management functions of the low-level CUDA driver application programming interface.

driver_get_version

Returns the latest CUDA version supported by driver.

Device Management

This section describes the device management functions of the low-level CUDA driver application programming interface.

device_get	Returns a handle to a compute device.
device_get_count	Returns the number of compute-capable devices.
device_get_name	Returns an identifier string for the device.
device_get_uuid_v2	Return an UUID for the device.
device_get_luid	Return an LUID and device node mask for the device.
device_total_mem_v2	Returns the total amount of memory on the device.
device_get_texture_1d_linear_max_width	Returns the maximum number of elements allocatable in a 1D linear texture for a given texture element size.
device_get_attribute	Returns information about the device.
device_get_properties	Returns properties for a selected device.
device_compute_capability	Returns the compute capability of the device.
device_get_host_atomic_capabilities	Query atomic operation capabilities between host and device.
device_get_nv_sci_sync_attributes	Fill in NvSciSync attributes for the given CUDA device.
device_set_mem_pool	Sets the current memory pool of a device.
device_get_mem_pool	Gets the current mempool for a device.
device_get_default_mem_pool	Returns the default mempool of a device.
device_get_exec_affinity_support	Returns information about the execution affinity support of the device.
flush_gpu_direct_rdma_writes	Blocks until remote writes are visible to the specified scope.

Primary Context Management

This section describes the primary context management functions of the low-level CUDA driver application programming interface.

The primary context is unique per device and shared with the CUDA runtime API. These functions allow integration with other libraries using CUDA.

device_primary_ctx_retain	Retain the primary context on the GPU.
device_primary_ctx_release_v2	Release the primary context on the GPU.
device_primary_ctx_set_flags_v2	Set flags for the primary context.
device_primary_ctx_get_state	Get the state of the primary context.
device_primary_ctx_reset_v2	Destroy all allocations and reset all state on the primary context.

Context Management

This section describes the context management functions of the low-level CUDA driver application programming interface.

Please note that some functions are described in Primary Context Management section.

ctx_create_v4	Create a CUDA context.
ctx_destroy_v2	Destroy a CUDA context.
ctx_push_current_v2	Pushes a context on the current CPU thread.
ctx_pop_current_v2	Pops the current CUDA context from the current CPU thread.
ctx_set_current	Binds the specified CUDA context to the calling CPU thread.
ctx_get_current	Returns the CUDA context bound to the calling CPU thread.
ctx_get_device	Returns the device handle for the current context.
ctx_get_device_v2	Returns the device handle for the specified context.
ctx_get_flags	Returns the flags for the current context.
ctx_set_flags	Sets the flags for the current context.
ctx_get_id	Returns the unique Id associated with the context supplied.
ctx_synchronize	Block for the current context's tasks to complete.
ctx_synchronize_v2	Block for the specified context's tasks to complete.
ctx_set_limit	Set resource limits.
ctx_get_limit	Returns resource limits.
ctx_get_cache_config	Returns the preferred cache configuration for the current context.
ctx_set_cache_config	Sets the preferred cache configuration for the current context.
ctx_get_api_version	Gets the context's API version.
ctx_get_stream_priority_range	Returns numerical values that correspond to the least and greatest stream priorities.
ctx_reset_persisting_l2cache	Resets all persisting lines in cache to normal status.
ctx_get_exec_affinity	Returns the execution affinity setting for the current context.
ctx_record_event	Records an event.
ctx_wait_event	Make a context wait on an event.

Module Management

This section describes the module management functions of the low-level CUDA driver application programming interface.

ModuleLoadingMode

CUDA Lazy Loading status

module_load	Loads a compute module.
module_load_data	Load a module's data.
module_load_data_ex	Load a module's data with options.
module_load_fat_binary	Load a module's data.
module_unload	Unloads a module.
module_get_loading_mode	Query lazy loading mode.
module_get_function	Returns a function handle.
module_get_function_count	Returns the number of functions within a module.
module_enumerate_functions	Returns the function handles within a module.
module_get_global_v2	Returns a global pointer from a module.
link_create_v2	Creates a pending JIT linker invocation.
link_add_data_v2	Add an input to a pending linker invocation.
link_add_file_v2	Add a file input to a pending linker invocation.
link_complete	Complete a pending linker invocation.
link_destroy	Destroys state for a JIT linker invocation.

Library Management

This section describes the library management functions of the low-level CUDA driver application programming interface.

library_load_data	Load a library with specified code and options.
library_load_from_file	Load a library with specified file and options.
library_unload	Unloads a library.
library_get_kernel	Returns a kernel handle.
library_get_kernel_count	Returns the number of kernels within a library.
library_enumerate_kernels	Retrieve the kernel handles within a library.
library_get_module	Returns a module handle.
kernel_get_function	Returns a function handle.
kernel_get_library	Returns a library handle.
library_get_global	Returns a global device pointer.
library_get_managed	Returns a pointer to managed memory.
library_get_unified_function	Returns a pointer to a unified function.
kernel_get_attribute	Returns information about a kernel.
kernel_set_attribute	Sets information about a kernel.
kernel_set_cache_config	Sets the preferred cache configuration for a device kernel.
kernel_get_name	Returns the function name for a CUkernel handle.
kernel_get_param_info	Returns the offset and size of a kernel parameter in the device-side parameter layout.
kernel_get_param_count	Returns the number of parameters used by the kernel.

Memory Management

This section describes the memory management functions of the low-level CUDA driver application programming interface.

MemDecompressParams	Empty-initialize an instance of CUmemDecompressParams.
MemDecompressAlgorithm	Bitmasks for CU_DEVICE_ATTRIBUTE_MEM_DECOMPRESS_ALGORITHM_MASK.

mem_get_info_v2	Gets free and total memory.
mem_alloc_v2	Allocates device memory.
mem_alloc_pitch_v2	Allocates pitched device memory.
mem_free_v2	Frees device memory.
mem_get_address_range_v2	Get information on memory allocations.
mem_alloc_host_v2	Allocates page-locked host memory.
mem_free_host	Frees page-locked host memory.
mem_host_alloc	Allocates page-locked host memory.
mem_host_get_device_pointer_v2	Passes back device pointer of mapped pinned memory.
mem_host_get_flags	Passes back flags that were used for a pinned allocation.
mem_alloc_managed	Allocates memory that will be automatically managed by the Unified Memory system.
device_register_async_notification	Registers a callback function to receive async notifications.
device_unregister_async_notification	Unregisters an async notification callback.
device_get_by_pci_bus_id	Returns a handle to a compute device.
device_get_pci_bus_id	Returns a PCI Bus Id string for the device.
ipc_get_event_handle	Gets an interprocess handle for a previously allocated event.
ipc_open_event_handle	Opens an interprocess event handle for use in the current process.
ipc_get_mem_handle	Gets an interprocess memory handle for an existing device memory allocation.
ipc_open_mem_handle_v2	Opens an interprocess memory handle exported from another process and returns a device pointer usable in the local process.
ipc_close_mem_handle	Attempts to close memory mapped with cuIpcOpenMemHandle.
mem_host_register_v2	Registers an existing host memory range for use by CUDA.
mem_host_unregister	Unregisters a memory range that was registered with cuMemHostRegister.
cu_memcpy	Copies memory.
memcpy_peer	Copies device memory between two contexts.
memcpy_htod_v2	Copies memory from Host to Device.
memcpy_dtoh_v2	Copies memory from Device to Host.
memcpy_dtod_v2	Copies memory from Device to Device.
memcpy_dtoa_v2	Copies memory from Device to Array.
memcpy_atod_v2	Copies memory from Array to Device.
memcpy_htoa_v2	Copies memory from Host to Array.
memcpy_atoh_v2	Copies memory from Array to Host.
memcpy_atoa_v2	Copies memory from Array to Array.
memcpy_2d_v2	Copies memory for 2D arrays.
memcpy_2d_unaligned_v2	Copies memory for 2D arrays.
memcpy_3d_v2	Copies memory for 3D arrays.
memcpy_3d_peer	Copies memory between contexts.
memcpy_async	Copies memory asynchronously.
memcpy_peer_async	Copies device memory between two contexts asynchronously.
memcpy_htod_async_v2	Copies memory from Host to Device.
memcpy_dtoh_async_v2	Copies memory from Device to Host.
memcpy_dtod_async_v2	Copies memory from Device to Device.
memcpy_htoa_async_v2	Copies memory from Host to Array.
memcpy_atoh_async_v2	Copies memory from Array to Host.
memcpy_2d_async_v2	Copies memory for 2D arrays.
memcpy_3d_async_v2	Copies memory for 3D arrays.
memcpy_3d_peer_async	Copies memory between contexts asynchronously.
memcpy_batch_async	Perform a batch of memory copies asynchronously.
memcpy_batch_async_v2	Performs a batch of memory copies asynchronously.
memcpy_3d_batch_async_v2	Performs a batch of 3D memory copies asynchronously.
memcpy_with_attributes_async	Performs asynchronous memory copy operation with the specified attributes.
memcpy_3d_with_attributes_async	Performs 3D memory copy with attributes asynchronously.
memset_d8_v2	Initializes device memory.
memset_d16_v2	Initializes device memory.
memset_d32_v2	Initializes device memory.
memset_d2d8_v2	Initializes device memory.
memset_d2d16_v2	Initializes device memory.
memset_d2d32_v2	Initializes device memory.
memset_d8_async	Sets device memory.
memset_d16_async	Sets device memory.
memset_d32_async	Sets device memory.
memset_d2d8_async	Sets device memory.
memset_d2d16_async	Sets device memory.
memset_d2d32_async	Sets device memory.
array_create_v2	Creates a 1D or 2D CUDA array.
array_get_descriptor_v2	Get a 1D or 2D CUDA array descriptor.
array_get_sparse_properties	Returns the layout properties of a sparse CUDA array.
mipmapped_array_get_sparse_properties	Returns the layout properties of a sparse CUDA mipmapped array.
array_get_memory_requirements	Returns the memory requirements of a CUDA array.
mipmapped_array_get_memory_requirements	Returns the memory requirements of a CUDA mipmapped array.
array_get_plane	Gets a CUDA array plane from a CUDA array.
array_destroy	Destroys a CUDA array.
array_3d_create_v2	Creates a 3D CUDA array.
array_3d_get_descriptor_v2	Get a 3D CUDA array descriptor.
mipmapped_array_create	Creates a CUDA mipmapped array.
mipmapped_array_get_level	Gets a mipmap level of a CUDA mipmapped array.
mipmapped_array_destroy	Destroys a CUDA mipmapped array.
mem_get_handle_for_address_range	Return a handle (a dma-buf file descriptor) for an address range.
mem_batch_decompress_async	Submit a batch of count independent decompression operations.

Virtual Memory Management

This section describes the virtual memory management functions of the low-level CUDA driver application programming interface.

mem_address_reserve	Allocate an address range reservation.
mem_address_free	Free an address range reservation.
mem_create	Create a CUDA memory handle representing a memory allocation of a given size described by the given properties.
mem_release	Release a memory handle representing a memory allocation which was previously allocated through cuMemCreate.
mem_map	Maps an allocation handle to a reserved virtual address range.
mem_map_array_async	Maps or unmaps subregions of sparse CUDA arrays and sparse CUDA mipmapped arrays.
mem_unmap	Unmap the backing memory of a given address range.
mem_set_access	Set the access flags for each location specified in desc for the given virtual address range.
mem_get_access	Get the access flags set for the given location and ptr.
mem_export_to_shareable_handle	Export an allocation to an OS-specific shareable handle.
mem_import_from_shareable_handle	Imports an allocation from a requested shareable handle type.
mem_get_allocation_granularity	Calculates either the minimal or recommended granularity.
mem_get_allocation_properties_from_handle	Retrieve the contents of the property structure defining properties for this handle.
mem_retain_allocation_handle	Given an address addr, returns the allocation handle of the backing memory allocation.

Stream Ordered Memory Allocator

This section describes the stream ordered memory allocator exposed by the low-level CUDA driver application programming interface.

overview

The asynchronous allocator allows the user to allocate and free in stream order. All asynchronous accesses of the allocation must happen between the stream executions of the allocation and the free. If the memory is accessed outside of the promised stream order, a use before allocation / use after free error will cause undefined behavior.

The allocator is free to reallocate the memory as long as it can guarantee that compliant memory accesses will not overlap temporally. The allocator may refer to internal stream ordering as well as inter-stream dependencies (such as CUDA events and null stream dependencies) when establishing the temporal guarantee. The allocator may also insert inter-stream dependencies to establish the temporal guarantee.

Supported Platforms

Whether or not a device supports the integrated stream ordered memory allocator may be queried by calling cuDeviceGetAttribute() with the device attribute CU_DEVICE_ATTRIBUTE_MEMORY_POOLS_SUPPORTED

mem_free_async	Frees memory with stream ordered semantics.
mem_alloc_async	Allocates memory with stream ordered semantics.
mem_pool_trim_to	Tries to release memory back to the OS.
mem_pool_set_attribute	Sets attributes of a memory pool.
mem_pool_get_attribute	Return an attribute of a memory pool (an int).
mem_pool_set_access	Controls visibility of pools between devices.
mem_pool_get_access	Returns the accessibility of a pool from a device.
mem_pool_create	Creates a memory pool.
mem_pool_destroy	Destroys the specified memory pool.
mem_get_default_mem_pool	Returns the default memory pool for a given location and allocation type.
mem_get_mem_pool	Gets the current memory pool for a memory location and of a particular allocation type.
mem_set_mem_pool	Sets the current memory pool for a memory location and allocation type.
mem_alloc_from_pool_async	Allocates memory from a specified pool with stream ordered semantics.
mem_pool_export_to_shareable_handle	Export a memory pool to an OS-specific shareable handle.
mem_pool_import_from_shareable_handle	imports a memory pool from a shared handle.
mem_pool_export_pointer	Export data to share a memory pool allocation between processes.
mem_pool_import_pointer	Import a memory pool allocation from another process.

Multicast Object Management

This section describes the CUDA multicast object operations exposed by the low-level CUDA driver application programming interface.

overview

A multicast object created via cuMulticastCreate enables certain memory operations to be broadcast to a team of devices. Devices can be added to a multicast object via cuMulticastAddDevice. Memory can be bound on each participating device via cuMulticastBindMem, cuMulticastBindMem_v2, cuMulticastBindAddr, or cuMulticastBindAddr_v2. Multicast objects can be mapped into a device’s virtual address space using the virtual memmory management APIs (see cuMemMap and cuMemSetAccess).

Supported Platforms

Support for multicast on a specific device can be queried using the device attribute CU_DEVICE_ATTRIBUTE_MULTICAST_SUPPORTED

multicast_create	Create a generic allocation handle representing a multicast object described by the given properties.
multicast_add_device	Associate a device to a multicast object.
multicast_bind_mem	Bind a memory allocation represented by a handle to a multicast object.
multicast_bind_mem_v2	Bind a memory allocation represented by a handle to a multicast object.
multicast_bind_addr	Bind a memory allocation represented by a virtual address to a multicast object.
multicast_bind_addr_v2	Bind a memory allocation represented by a virtual address to a multicast object.
multicast_unbind	Unbind any memory allocations bound to a multicast object at a given offset and upto a given size.
multicast_get_granularity	Calculates either the minimal or recommended granularity for multicast object.

Fabric Clique Information

This section describes functions and data structures to retrieve fabric clique information.

CliqueInfo	Empty-initialize an array of CUcliqueInfo.
CliqueType	Fabric clique types

device_get_fabric_cluster_uuid	Retrieves the fabric cluster UUID.
device_get_clique_count	Retrieves the number of fabric cliques.
device_get_clique_info	Retrieves fabric clique information.

Logical Endpoint

This section describes the logical endpoint functions of the low-level CUDA driver application programming interface.

LogicalEndpointFabricHandle	Empty-initialize an instance of CUlogicalEndpointFabricHandle.
LogicalEndpointProp	Empty-initialize an instance of CUlogicalEndpointProp.
LogicalEndpointIpcHandleType	IPC handle types that can be requested/queried for a given logical endpoint
LogicalEndpointType	Logical endpoint type
LogicalEndpointFlag	Flags for CUlogicalEndpointProp

logical_endpoint_id_reserve	Reserves a range of logical endpoint ids.
logical_endpoint_id_release	Releases a range of logical endpoint ids.
logical_endpoint_create	Creates a logical endpoint with the requested properties and associates it with the logical endpoint id.
logical_endpoint_add_device	Associates a device to a multicast logical endpoint.
logical_endpoint_destroy	Removes the association of the logical endpoint from the logical endpoint id.
logical_endpoint_bind_addr	Bind a memory allocation represented by a virtual address to a logical endpoint.
logical_endpoint_bind_mem	Binds memory object represented by a handle to the logical endpoint.
logical_endpoint_unbind	Unbinds any binding at offset from the logical endpoint.
logical_endpoint_export	Export a logical endpoint to an IPC handle.
logical_endpoint_import	Imports a logical endpoint from the given IPC handle and associates it with a logical endpoint id.
logical_endpoint_get_limits	Calculates the minimum alignment and the maximum size for the given logical endpoint properties.
logical_endpoint_query	Determines if all logical endpoints in the range have been successfully constructed.

Unified Addressing

This section describes the unified addressing functions of the low-level CUDA driver application programming interface.

Overview

CUDA devices can share a unified address space with the host. For these devices there is no distinction between a device pointer and a host pointer – the same pointer value may be used to access memory from the host program and from a kernel running on the device (with exceptions enumerated below).

Supported Platforms

Whether or not a device supports unified addressing may be queried by calling cuDeviceGetAttribute() with the device attribute CU_DEVICE_ATTRIBUTE_UNIFIED_ADDRESSING.

Unified addressing is automatically enabled in 64-bit processes

Looking Up Information from Pointer Values

It is possible to look up information about the memory which backs a pointer value. For instance, one may want to know if a pointer points to host or device memory. As another example, in the case of device memory, one may want to know on which CUDA device the memory resides. These properties may be queried using the function cuPointerGetAttribute()

Since pointers are unique, it is not necessary to specify information about the pointers specified to the various copy functions in the CUDA API. The function cuMemcpy() may be used to perform a copy between two pointers, ignoring whether they point to host or device memory (making cuMemcpyHtoD(), cuMemcpyDtoD(), and cuMemcpyDtoH() unnecessary for devices supporting unified addressing). For multidimensional copies, the memory type CU_MEMORYTYPE_UNIFIED may be used to specify that the CUDA driver should infer the location of the pointer from its value.

Automatic Mapping of Host Allocated Host Memory

All host memory allocated in all contexts using cuMemAllocHost() and cuMemHostAlloc() is always directly accessible from all contexts on all devices that support unified addressing. This is the case regardless of whether or not the flags CU_MEMHOSTALLOC_PORTABLE and CU_MEMHOSTALLOC_DEVICEMAP are specified.

The pointer value through which allocated host memory may be accessed in kernels on all devices that support unified addressing is the same as the pointer value through which that memory is accessed on the host, so it is not necessary to call cuMemHostGetDevicePointer() to get the device pointer for these allocations.

Note that this is not the case for memory allocated using the flag CU_MEMHOSTALLOC_WRITECOMBINED, as discussed below.

Automatic Registration of Peer Memory

Upon enabling direct access from a context that supports unified addressing to another peer context that supports unified addressing using cuCtxEnablePeerAccess() all memory allocated in the peer context using cuMemAlloc() and cuMemAllocPitch() will immediately be accessible by the current context. The device pointer value through which any peer memory may be accessed in the current context is the same pointer value through which that memory may be accessed in the peer context.

Exceptions, Disjoint Addressing

Not all memory may be accessed on devices through the same pointer value through which they are accessed on the host. These exceptions are host memory registered using cuMemHostRegister() and host memory allocated using the flag CU_MEMHOSTALLOC_WRITECOMBINED. For these exceptions, there exists a distinct host and device address for the memory. The device address is guaranteed to not overlap any valid host pointer range and is guaranteed to have the same value across all contexts that support unified addressing.

This device address may be queried using cuMemHostGetDevicePointer() when a context using unified addressing is current. Either the host or the unified device pointer value may be used to refer to this memory through cuMemcpy() and similar functions using the CU_MEMORYTYPE_UNIFIED memory type.

pointer_get_attribute	Return an attribute of a pointer.
mem_prefetch_async_v2	Prefetches memory to the specified destination location.
mem_advise_v2	Advise about the usage of a given memory range.
mem_prefetch_batch_async	Performs a batch of memory prefetches asynchronously.
mem_discard_batch_async	Performs a batch of memory discards asynchronously.
mem_discard_and_prefetch_batch_async	Performs a batch of memory discards and prefetches asynchronously.
mem_get_location_info	Gets residency information for a memory address range.
mem_range_get_attribute	Return an attribute of a memory range.
mem_range_get_attributes	Return several attributes of a memory range, in one call.
pointer_set_attribute	Set attributes on a previously allocated memory region.
pointer_get_attributes	Return several attributes of a pointer, in one call.

Stream Management

This section describes the stream management functions of the low-level CUDA driver application programming interface.

GraphRecaptureStatus

See CUgraphRecaptureStatus.

stream_create	Create a stream.
stream_create_with_priority	Create a stream with the given priority.
stream_begin_capture_to_cig	Begins capture to CIG on a stream.
stream_end_capture_to_cig	Ends CIG capture on a stream.
stream_get_priority	Query the priority of a given stream.
stream_get_device	Returns the device handle of the stream.
stream_get_flags	Query the flags of a given stream.
stream_get_id	Returns the unique Id associated with the stream handle supplied.
stream_get_ctx	Query the context associated with a stream.
stream_get_ctx_v2	Query the contexts associated with a stream.
stream_wait_event	Make a compute stream wait on an event.
stream_add_callback	Add a callback to a compute stream.
stream_begin_capture_v2	Begins graph capture on a stream.
stream_begin_recapture_to_graph	Begin graph capture on a stream to an existing graph.
stream_begin_capture_to_graph	Begins graph capture on a stream to an existing graph.
thread_exchange_stream_capture_mode	Swap the stream capture interaction mode of the calling thread.
stream_end_capture	Ends capture on a stream, returning the captured graph.
stream_is_capturing	Returns a stream's capture status.
stream_get_capture_info_v2	Query a stream's capture state.
stream_get_capture_info_v3	Query a stream's capture state.
stream_update_capture_dependencies_v2	Update the set of dependencies in a capturing stream.
stream_attach_mem_async	Attach memory to a stream asynchronously.
stream_query	Determine status of a compute stream.
stream_synchronize	Wait until a stream's tasks are completed.
stream_destroy_v2	Destroys a stream.
stream_copy_attributes	Copies attributes from source stream to destination stream.
stream_get_attribute	Queries stream attribute.
stream_set_attribute	Sets stream attribute.

Event Management

This section describes the event management functions of the low-level CUDA driver application programming interface.

event_create	Creates an event.
event_record	Records an event.
event_record_with_flags	Records an event.
event_query	Queries an event's status.
event_synchronize	Waits for an event to complete.
event_destroy_v2	Destroys an event.
event_elapsed_time_v2	Computes the elapsed time between two events.

External Resource Interoperability

This section describes the external resource interoperability functions of the low-level CUDA driver application programming interface.

import_external_memory	Imports an external memory object.
external_memory_get_mapped_buffer	Maps a buffer onto an imported memory object.
external_memory_get_mapped_mipmapped_array	Maps a CUDA mipmapped array onto an external memory object.
destroy_external_memory	Destroys an external memory object.
import_external_semaphore	Imports an external semaphore.
signal_external_semaphores_async	Signals a set of external semaphore objects.
wait_external_semaphores_async	Waits on a set of external semaphore objects.
destroy_external_semaphore	Destroys an external semaphore.

Stream Memory Operations

This section describes the stream memory operations of the low-level CUDA driver application programming interface.

Support for the CU_STREAM_WAIT_VALUE_NOR flag can be queried with ::CU_DEVICE_ATTRIBUTE_CAN_USE_STREAM_WAIT_VALUE_NOR_V2.

Support for the cuStreamWriteValue64() and cuStreamWaitValue64() functions, as well as for the CU_STREAM_MEM_OP_WAIT_VALUE_64 and CU_STREAM_MEM_OP_WRITE_VALUE_64 flags, can be queried with CU_DEVICE_ATTRIBUTE_CAN_USE_64_BIT_STREAM_MEM_OPS.

Support for both CU_STREAM_WAIT_VALUE_FLUSH and CU_STREAM_MEM_OP_FLUSH_REMOTE_WRITES requires dedicated platform hardware features and can be queried with cuDeviceGetAttribute() and CU_DEVICE_ATTRIBUTE_CAN_FLUSH_REMOTE_WRITES.

Note that all memory pointers passed as parameters to these operations are device pointers. Where necessary a device pointer should be obtained, for example with cuMemHostGetDevicePointer().

None of the operations accepts pointers to managed memory buffers (cuMemAllocManaged).

Warning: Improper use of these APIs may deadlock the application. Synchronization ordering established through these APIs is not visible to CUDA. CUDA tasks that are (even indirectly) ordered by these APIs should also have that order expressed with CUDA-visible dependencies such as events. This ensures that the scheduler does not serialize them in an improper order.

stream_wait_value32_v2	Wait on a memory location.
stream_wait_value64_v2	Wait on a memory location.
stream_write_value32_v2	Write a value to memory.
stream_write_value64_v2	Write a value to memory.
stream_batch_mem_op_v2	Batch operations to synchronize the stream via memory operations.

Execution Control

This section describes the execution control functions of the low-level CUDA driver application programming interface.

FunctionLoadingState

See CUfunctionLoadingState.

func_get_attribute	Returns information about a function.
func_set_attribute	Sets information about a function.
func_set_cache_config	Sets the preferred cache configuration for a device function.
func_get_module	Returns a module handle.
func_get_name	Returns the function name for a CUfunction handle.
func_get_param_info	Returns the offset and size of a kernel parameter in the device-side parameter layout.
func_get_param_count	Returns the number of parameters used by the function.
func_is_loaded	Returns if the function is loaded.
func_load	Loads a function.
launch_kernel	Launches a CUDA function CUfunction or a CUDA kernel CUkernel.
launch_kernel_ex	Launches a CUDA function CUfunction or a CUDA kernel CUkernel with launch-time configuration.
launch_cooperative_kernel	Launches a CUDA function CUfunction or a CUDA kernel CUkernel where thread blocks can cooperate and synchronize as they execute.
launch_cooperative_kernel_multi_device	Launches CUDA functions on multiple devices where thread blocks can cooperate and synchronize as they execute.
launch_host_func	Enqueues a host function call in a stream.
launch_host_func_v2	Enqueues a host function call in a stream.

Graph Management

This section describes the graph management functions of the low-level CUDA driver application programming interface.

graph_create	Creates a graph.
graph_add_kernel_node_v2	Creates a kernel execution node and adds it to a graph.
graph_kernel_node_get_params_v2	Returns a kernel node's parameters.
graph_kernel_node_set_params_v2	Sets a kernel node's parameters.
graph_add_memcpy_node	Creates a memcpy node and adds it to a graph.
graph_memcpy_node_get_params	Returns a memcpy node's parameters.
graph_memcpy_node_set_params	Sets a memcpy node's parameters.
graph_add_memset_node	Creates a memset node and adds it to a graph.
graph_memset_node_get_params	Returns a memset node's parameters.
graph_memset_node_set_params	Sets a memset node's parameters.
graph_add_host_node	Creates a host execution node and adds it to a graph.
graph_host_node_get_params	Returns a host node's parameters.
graph_host_node_set_params	Sets a host node's parameters.
graph_add_child_graph_node	Creates a child graph node and adds it to a graph.
graph_child_graph_node_get_graph	Gets a handle to the embedded graph of a child graph node.
graph_add_empty_node	Creates an empty node and adds it to a graph.
graph_add_event_record_node	Creates an event record node and adds it to a graph.
graph_event_record_node_get_event	Returns the event associated with an event record node.
graph_event_record_node_set_event	Sets an event record node's event.
graph_add_event_wait_node	Creates an event wait node and adds it to a graph.
graph_event_wait_node_get_event	Returns the event associated with an event wait node.
graph_event_wait_node_set_event	Sets an event wait node's event.
graph_add_external_semaphores_signal_node	Creates an external semaphore signal node and adds it to a graph.
graph_external_semaphores_signal_node_get_params	Returns an external semaphore signal node's parameters.
graph_external_semaphores_signal_node_set_params	Sets an external semaphore signal node's parameters.
graph_add_external_semaphores_wait_node	Creates an external semaphore wait node and adds it to a graph.
graph_external_semaphores_wait_node_get_params	Returns an external semaphore wait node's parameters.
graph_external_semaphores_wait_node_set_params	Sets an external semaphore wait node's parameters.
graph_add_batch_mem_op_node	Creates a batch memory operation node and adds it to a graph.
graph_batch_mem_op_node_get_params	Returns a batch mem op node's parameters.
graph_batch_mem_op_node_set_params	Sets a batch mem op node's parameters.
graph_exec_batch_mem_op_node_set_params	Sets the parameters for a batch mem op node in the given graphExec.
graph_add_mem_alloc_node	Creates an allocation node and adds it to a graph.
graph_mem_alloc_node_get_params	Returns a memory alloc node's parameters.
graph_add_mem_free_node	Creates a memory free node and adds it to a graph.
graph_mem_free_node_get_params	Returns a memory free node's parameters.
device_graph_mem_trim	Free unused memory that was cached on the specified device for use with graphs back to the OS.
device_get_graph_mem_attribute	Return a graph memory attribute of a device, in bytes.
device_set_graph_mem_attribute	Set asynchronous allocation attributes related to graphs.
graph_clone	Clones a graph.
graph_node_find_in_clone	Finds a cloned version of a node.
graph_node_get_type	Returns a node's type.
graph_node_get_containing_graph	Returns the graph that contains a given graph node.
graph_node_get_local_id	Returns the local node id of a given graph node.
graph_node_get_tools_id	Returns an id used by tools to identify a given node.
graph_get_id	Returns the id of a given graph.
graph_exec_get_id	Returns the id of a given graph exec.
graph_get_nodes	Returns a graph's nodes.
graph_get_root_nodes	Returns a graph's root nodes.
graph_get_edges	graph_get_edges_v2(intptr_t h_graph) -> tuple
graph_get_edges_v2	Returns a graph's dependency edges.
graph_node_get_dependencies	graph_node_get_dependencies_v2(intptr_t h_node) -> tuple
graph_node_get_dependencies_v2	Returns a node's dependencies.
graph_node_get_dependent_nodes	graph_node_get_dependent_nodes_v2(intptr_t h_node) -> tuple
graph_node_get_dependent_nodes_v2	Returns a node's dependent nodes.
graph_add_dependencies_v2	Adds dependency edges to a graph.
graph_remove_dependencies_v2	Removes dependency edges from a graph.
graph_destroy_node	Remove a node from the graph.
graph_instantiate	graph_instantiate_with_flags(intptr_t h_graph, unsigned long long flags) -> intptr_t
graph_instantiate_with_flags	Creates an executable graph from a graph.
graph_instantiate_with_params	Creates an executable graph from a graph.
graph_exec_get_flags	Query the instantiation flags of an executable graph.
graph_exec_kernel_node_set_params_v2	Sets the parameters for a kernel node in the given graphExec.
graph_exec_memcpy_node_set_params	Sets the parameters for a memcpy node in the given graphExec.
graph_exec_memset_node_set_params	Sets the parameters for a memset node in the given graphExec.
graph_exec_host_node_set_params	Sets the parameters for a host node in the given graphExec.
graph_exec_child_graph_node_set_params	Updates node parameters in the child graph node in the given graphExec.
graph_exec_event_record_node_set_event	Sets the event for an event record node in the given graphExec.
graph_exec_event_wait_node_set_event	Sets the event for an event wait node in the given graphExec.
graph_exec_external_semaphores_signal_node_set_params	Sets the parameters for an external semaphore signal node in the given graphExec.
graph_exec_external_semaphores_wait_node_set_params	Sets the parameters for an external semaphore wait node in the given graphExec.
graph_node_set_enabled	Enables or disables the specified node in the given graphExec.
graph_node_get_enabled	Query whether a node in the given graphExec is enabled.
graph_upload	Uploads an executable graph in a stream.
graph_launch	Launches an executable graph in a stream.
graph_exec_destroy	Destroys an executable graph.
graph_destroy	Destroys a graph.
graph_exec_update_v2	Check whether an executable graph can be updated with a graph and perform the update if possible.
graph_kernel_node_copy_attributes	Copies attributes from source node to destination node.
graph_kernel_node_get_attribute	Queries node attribute.
graph_kernel_node_set_attribute	Sets node attribute.
graph_debug_dot_print	Write a DOT file describing graph structure.
user_object_create	Create a user object.
user_object_retain	Retain a reference to a user object.
user_object_release	Release a reference to a user object.
graph_retain_user_object	Retain a reference to a user object from a graph.
graph_release_user_object	Release a user object reference from a graph.
graph_add_node_v2	Adds a node of arbitrary type to a graph.
graph_add_node_v3	Adds a node of arbitrary type to a graph.
graph_node_set_params	Update a graph node's parameters.
graph_node_set_params_v2	Update a graph node's parameters.
graph_node_get_params	Return a graph node's parameters.
graph_exec_node_set_params	Update a graph node's parameters in an instantiated graph.
graph_conditional_handle_create	Create a conditional handle.

Occupancy

This section describes the occupancy calculation functions of the low-level CUDA driver application programming interface.

occupancy_max_active_blocks_per_multiprocessor	Returns occupancy of a function.
occupancy_max_active_blocks_per_multiprocessor_with_flags	Returns occupancy of a function.
occupancy_max_potential_block_size	Suggest a launch configuration with reasonable occupancy.
occupancy_max_potential_block_size_with_flags	Suggest a launch configuration with reasonable occupancy.
occupancy_available_dynamic_smem_per_block	Returns dynamic shared memory available per block when launching num_blocks blocks on SM.
occupancy_max_potential_cluster_size	Given the kernel function (func) and launch configuration (config), return the maximum cluster size in *cluster_size.
occupancy_max_active_clusters	Given the kernel function (func) and launch configuration (config), return the maximum number of clusters that could co-exist on the target device in *num_clusters.

Texture Object Management

This section describes the texture object management functions of the low-level CUDA driver application programming interface. The texture object API is only supported on devices of compute capability 3.0 or higher.

tex_object_create	Creates a texture object.
tex_object_destroy	Destroys a texture object.
tex_object_get_resource_desc	Returns a texture object's resource descriptor.
tex_object_get_texture_desc	Returns a texture object's texture descriptor.
tex_object_get_resource_view_desc	Returns a texture object's resource view descriptor.

Surface Object Management

This section describes the surface object management functions of the low-level CUDA driver application programming interface. The surface object API is only supported on devices of compute capability 3.0 or higher.

surf_object_create	Creates a surface object.
surf_object_destroy	Destroys a surface object.
surf_object_get_resource_desc	Returns a surface object's resource descriptor.

Tensor Map Object Managment

This section describes the tensor map object management functions of the low-level CUDA driver application programming interface. The tensor core API is only supported on devices of compute capability 9.0 or higher.

tensor_map_encode_tiled	Create a tensor map descriptor object representing tiled memory region.
tensor_map_encode_im2col	Create a tensor map descriptor object representing im2col memory region.
tensor_map_encode_im2col_wide	Create a tensor map descriptor object representing im2col memory region, but where the elements are exclusively loaded along the W dimension.
tensor_map_replace_address	Modify an existing tensor map descriptor with an updated global address.

Peer Context Memory Access

This section describes the direct peer context memory access functions of the low-level CUDA driver application programming interface.

device_can_access_peer	Queries if a device may directly access a peer device's memory.
ctx_enable_peer_access	Enables direct access to memory allocations in a peer context.
ctx_disable_peer_access	Disables direct access to memory allocations in a peer context and unregisters any registered allocations.
device_get_p2p_attribute	Queries attributes of the link between two devices.
device_get_p2p_atomic_capabilities	Queries details about atomic operations supported between two devices.

Graphics Interoperability

This section describes the graphics interoperability functions of the low-level CUDA driver application programming interface.

graphics_unregister_resource	Unregisters a graphics resource for access by CUDA.
graphics_sub_resource_get_mapped_array	Get an array through which to access a subresource of a mapped graphics resource.
graphics_resource_get_mapped_mipmapped_array	Get a mipmapped array through which to access a mapped graphics resource.
graphics_resource_get_mapped_pointer_v2	Get a device pointer through which to access a mapped graphics resource.
graphics_resource_set_map_flags_v2	Set usage flags for mapping a graphics resource.
graphics_map_resources	Map graphics resources for access by CUDA.
graphics_unmap_resources	Unmap graphics resources.

Driver Entry Point Access

This section describes the driver entry point access functions of the low-level CUDA driver application programming interface.

get_proc_address_v2	Returns the requested driver API function pointer.
get_export_table

Coredump Attributes Control API

This section describes the coredump attribute control functions of the low-level CUDA driver application programming interface.

CoredumpSettings	Flags for choosing a coredump attribute to get/set
CoredumpGenerationFlags	Flags for controlling coredump contents

coredump_get_attribute	Fetch a coredump attribute value for the current context.
coredump_get_attribute_global	Fetch a coredump attribute value for the entire application.
coredump_set_attribute	Set a coredump attribute value for the current context.
coredump_set_attribute_global	Set a coredump attribute value globally.
coredump_register_start_callback	Register a callback to be invoked when a GPU coredump begins.
coredump_register_complete_callback	Register a callback to be invoked when a GPU coredump completes.
coredump_deregister_start_callback	Deregister a previously registered coredump start callback.
coredump_deregister_complete_callback	Deregister a previously registered coredump complete callback.

Green Contexts

This section describes the APIs for creation and manipulation of green contexts in the CUDA driver. Green contexts are a lightweight alternative to traditional contexts, that can be used to select a subset of device resources. This allows the developer to, for example, select SMs from distinct spatial partitions of the GPU and target them via CUDA stream operations, kernel launches, etc.

Here are the broad initial steps to follow to get started:

• 1. Start with an initial set of resources. For SM resources, they can be fetched via cuDeviceGetDevResource. In case of workqueues, a new configuration can be used or an existing one queried via the cuDeviceGetDevResource API.

• 2. Modify these resources by either partitioning them (in case of SMs) or changing the configuration (in case of workqueues). To partition SMs, we recommend cuDevSmResourceSplit. Changing the workqueue configuration can be done directly in place.

• 3. Finalize the specification of resources by creating a descriptor via cuDevResourceGenerateDesc.

• 4. Create a green context via cuGreenCtxCreate. This provisions the resource, such as workqueues (until this step it was only a configuration specification).

• 5. Create a stream via cuGreenCtxStreamCreate, and use it throughout your application.

SMs

There are two possible partition operations - with cuDevSmResourceSplitByCount the partitions created have to follow default SM count granularity requirements, so it will often be rounded up and aligned to a default value. On the other hand, cuDevSmResourceSplit is explicit and allows for creation of non-equal groups. It will not round up automatically - instead it is the developer’s responsibility to query and set the correct values. These requirements can be queried with cuDeviceGetDevResource to determine the alignment granularity (sm.smCoscheduledAlignment). A general guideline on the default values for each compute architecture:

• On all architectures,

  • Portable code should set smCount to a multiple of the device’s alignment granularity (sm.smCoscheduledAlignment).

• On Compute Architecture 7.X, 8.X, and all Tegra SoC:

  • The smCount must be a multiple of 2.

  • The alignment (and default value of coscheduledSmCount) is 2.

• On Compute Architecture 9.0+:

  • The smCount must be a multiple of coscheduledSmCount if provided.

  • The alignment is 8.

While the maximum value for coscheduled SM count is 32 on all Compute Architecture 9.0+, it’s recommended to follow cluster size requirements. The portable cluster size and the max cluster size should be used in order to benefit from this co-scheduling.

Workqueues

For CU_DEV_RESOURCE_TYPE_WORKQUEUE_CONFIG, the resource specifies the expected maximum number of concurrent stream-ordered workloads via the wqConcurrencyLimit field. The sharingScope field determines how workqueue resources are shared:

• CU_WORKQUEUE_SCOPE_DEVICE_CTX: Use all shared workqueue resources across all contexts (default driver behavior).

• CU_WORKQUEUE_SCOPE_GREEN_CTX_BALANCED: When possible, use non-overlapping workqueue resources with other balanced green contexts.

The maximum concurrency limit depends on ::CUDA_DEVICE_MAX_CONNECTIONS and can be queried from the primary context via cuCtxGetDevResource. Configurations may exceed this concurrency limit, but the driver will not guarantee that work submission remains non-overlapping.

For CU_DEV_RESOURCE_TYPE_WORKQUEUE, the resource represents a pre-existing workqueue that can be retrieved from existing contexts or green contexts. This allows reusing workqueue resources across different green contexts.

On Concurrency

Even if the green contexts have disjoint SM partitions, it is not guaranteed that the kernels launched in them will run concurrently or have forward progress guarantees. This is due to other resources that could cause a dependency. Using a combination of disjoint SMs and CU_WORKQUEUE_SCOPE_GREEN_CTX_BALANCED workqueue configurations can provide the best chance of avoiding interference. More resources will be added in the future to provide stronger guarantees.

Additionally, there are two known scenarios, where its possible for the workload to run on more SMs than was provisioned (but never less).

• On Volta+ MPS: When CUDA_MPS_ACTIVE_THREAD_PERCENTAGE is used, the set of SMs that are used for running kernels can be scaled up to the value of SMs used for the MPS client.

• On Compute Architecture 9.x: When a module with dynamic parallelism (CDP) is loaded, all future kernels running under green contexts may use and share an additional set of 2 SMs.

Memory Copy Operations

Green context restrictions apply to memory copy operations only when the copy is performed using a green context. For cross-device copies, green context restrictions may not be applied.

DevSmResource	Empty-initialize an instance of CUdevSmResource.
DevWorkqueueConfigResource	Empty-initialize an instance of CUdevWorkqueueConfigResource.
DevWorkqueueResource	Empty-initialize an instance of CUdevWorkqueueResource.
DevResource_v1	Empty-initialize an array of CUdevResource_v1.
GreenCtxCreateFlags	Flags for green context creation
DevSmResourceGroupFlags	Flags for a CUdevSmResource group
DevSmResourceSplitByCountFlags	See CUdevSmResourceSplitByCount_flags.
DevResourceType	Type of resource
DevWorkqueueConfigScope	Sharing scope for workqueues

green_ctx_create	Creates a green context with a specified set of resources.
green_ctx_destroy	Destroys a green context.
ctx_from_green_ctx	Returns a CUcontext handle for a green context.
device_get_dev_resource	Get device resources.
ctx_get_dev_resource	Get context resources.
green_ctx_get_dev_resource	Get green context resources.
dev_sm_resource_split_by_count	Splits CU_DEV_RESOURCE_TYPE_SM resources.
dev_sm_resource_split	Splits a CU_DEV_RESOURCE_TYPE_SM resource into structured groups.
dev_resource_generate_desc	Generate a resource descriptor.
green_ctx_record_event	Records an event.
green_ctx_wait_event	Make a green context wait on an event.
stream_get_green_ctx	Query the green context associated with a stream.
green_ctx_stream_create	Create a stream for use in the green context.
green_ctx_get_id	Returns the unique Id associated with the green context supplied.
stream_get_dev_resource	Get stream resources.

Error Log Management Functions

This section describes the error log management functions of the low-level CUDA driver application programming interface.

LogLevel

See CUlogLevel.

logs_register_callback	Register a callback function to receive error log messages.
logs_unregister_callback	Unregister a log message callback.
logs_current	Sets log iterator to point to the end of log buffer, where the next message would be written.
logs_dump_to_file	Dump the driver's error log buffer to a file.
logs_dump_to_memory	Dump accumulated driver logs into a buffer.

CUDA Checkpointing

CUDA API versioning support

This sections describes the checkpoint and restore functions of the low-level CUDA driver application programming interface.

The CUDA checkpoint and restore API’s provide a way to save and restore GPU state for full process checkpoints when used with CPU side process checkpointing solutions. They can also be used to pause GPU work and suspend a CUDA process to allow other applications to make use of GPU resources.

Checkpoint and restore capabilities are currently restricted to Linux.

checkpoint_process_get_restore_thread_id	Returns the restore thread ID for a CUDA process.
checkpoint_process_get_state	Returns the process state of a CUDA process.
checkpoint_process_lock	Lock a running CUDA process.
checkpoint_process_checkpoint	Checkpoint a CUDA process's GPU memory contents.
checkpoint_process_restore	Restore a CUDA process's GPU memory contents from its last checkpoint.
checkpoint_operation_complete	Complete a custom-storage checkpoint or restore operation.
checkpoint_process_unlock	Unlock a CUDA process to allow CUDA API calls.

Profiler Control

This section describes the profiler control functions of the low-level CUDA driver application programming interface.

profiler_start	Enable profiling.
profiler_stop	Disable profiling.

EGL Interoperability

This section describes the EGL interoperability functions of the low-level CUDA driver application programming interface.

graphics_egl_register_image	Registers an EGL image.
egl_stream_consumer_connect	Connect CUDA to EGLStream as a consumer.
egl_stream_consumer_connect_with_flags	Connect CUDA to EGLStream as a consumer with given flags.
egl_stream_consumer_disconnect	Disconnect CUDA as a consumer to EGLStream .
egl_stream_consumer_acquire_frame	Acquire an image frame from the EGLStream with CUDA as a consumer.
egl_stream_consumer_release_frame	Releases the last frame acquired from the EGLStream.
egl_stream_producer_connect	Connect CUDA to EGLStream as a producer.
egl_stream_producer_disconnect	Disconnect CUDA as a producer to EGLStream .
egl_stream_producer_present_frame	Present a CUDA eglFrame to the EGL stream.
egl_stream_producer_return_frame	Return a CUDA eglFrame to the EGL stream released by the consumer.
graphics_resource_get_mapped_egl_frame	Get an EGL frame through which to access a registered EGL resource.
event_create_from_egl_sync	Creates an event from EGLSync object.

OpenGL Interoperability

This section describes the OpenGL interoperability functions of the low-level CUDA driver application programming interface. Note that mapping of OpenGL resources is performed with the graphics API agnostic, resource mapping interface described in Graphics Interoperability.

graphics_gl_register_buffer	Registers an OpenGL buffer object.
graphics_gl_register_image	Register an OpenGL texture or renderbuffer object.
gl_get_devices_v2	Gets the CUDA devices associated with the current OpenGL context.

VDPAU Interoperability

This section describes the VDPAU interoperability functions of the low-level CUDA driver application programming interface.

vdpau_get_device	Gets the CUDA device associated with a VDPAU device.
vdpau_ctx_create_v2	Create a CUDA context for interoperability with VDPAU.
graphics_vdpau_register_video_surface	Registers a VDPAU VdpVideoSurface object.
graphics_vdpau_register_output_surface	Registers a VDPAU VdpOutputSurface object.

Deprecated Context Management

This section describes the deprecated context management functions of the low-level CUDA driver application programming interface.

ctx_attach	Increment a context's usage-count.
ctx_detach	Decrement a context's usage-count.
ctx_get_shared_mem_config	Returns the current shared memory configuration for the current context.
ctx_set_shared_mem_config	Sets the shared memory configuration for the current context.

Deprecated Module Management

This section describes the deprecated module management functions of the low-level CUDA driver application programming interface.

module_get_tex_ref	Returns a handle to a texture reference.
module_get_surf_ref	Returns a handle to a surface reference.

Deprecated Execution Control

This section describes the deprecated execution control functions of the low-level CUDA driver application programming interface.

func_set_block_shape	Sets the block-dimensions for the function.
func_set_shared_size	Sets the dynamic shared-memory size for the function.
func_set_shared_mem_config	Sets the shared memory configuration for a device function.
param_set_size	Sets the parameter size for the function.
param_seti	Adds an integer parameter to the function's argument list.
param_setf	Adds a floating-point parameter to the function's argument list.
param_setv	Adds arbitrary data to the function's argument list.
param_set_tex_ref	Adds a texture-reference to the function's argument list.
launch	Launches a CUDA function.
launch_grid	Launches a CUDA function.
launch_grid_async	Launches a CUDA function.

Deprecated Texture Reference Management

This section describes the deprecated texture reference management functions of the low-level CUDA driver application programming interface.

tex_ref_create	Creates a texture reference.
tex_ref_destroy	Destroys a texture reference.
tex_ref_set_array	Binds an array as a texture reference.
tex_ref_set_mipmapped_array	Binds a mipmapped array to a texture reference.
tex_ref_set_address_v2	Binds an address as a texture reference.
tex_ref_set_address2d_v3	Binds an address as a 2D texture reference.
tex_ref_set_format	Sets the format for a texture reference.
tex_ref_set_address_mode	Sets the addressing mode for a texture reference.
tex_ref_set_filter_mode	Sets the filtering mode for a texture reference.
tex_ref_set_mipmap_filter_mode	Sets the mipmap filtering mode for a texture reference.
tex_ref_set_mipmap_level_bias	Sets the mipmap level bias for a texture reference.
tex_ref_set_mipmap_level_clamp	Sets the mipmap min/max mipmap level clamps for a texture reference.
tex_ref_set_max_anisotropy	Sets the maximum anisotropy for a texture reference.
tex_ref_set_border_color	Sets the border color for a texture reference.
tex_ref_set_flags	Sets the flags for a texture reference.
tex_ref_get_address_v2	Gets the address associated with a texture reference.
tex_ref_get_array	Gets the array bound to a texture reference.
tex_ref_get_mipmapped_array	Gets the mipmapped array bound to a texture reference.
tex_ref_get_address_mode	Gets the addressing mode used by a texture reference.
tex_ref_get_filter_mode	Gets the filter-mode used by a texture reference.
tex_ref_get_format	Gets the format used by a texture reference.
tex_ref_get_mipmap_filter_mode	Gets the mipmap filtering mode for a texture reference.
tex_ref_get_mipmap_level_bias	Gets the mipmap level bias for a texture reference.
tex_ref_get_mipmap_level_clamp	Gets the min/max mipmap level clamps for a texture reference.
tex_ref_get_max_anisotropy	Gets the maximum anisotropy for a texture reference.
tex_ref_get_border_color	Return the border color of a texture reference.
tex_ref_get_flags	Gets the flags used by a texture reference.

Deprecated Surface Reference Management

This section describes the deprecated surface reference management functions of the low-level CUDA driver application programming interface.

surf_ref_set_array	Sets the CUDA array for a surface reference.
surf_ref_get_array	Passes back the CUDA array bound to a surface reference.