Concurrency and Thread Safety#
Subject to the CUDA driver limitations below, cuda.core allows concurrent
reads of its objects from multiple host threads. Concurrent mutation of the
same object is not supported –
for example, adding nodes to the same graph, or closing a resource while
another thread is using it. Whenever an object is shared across threads and at
least one of them may mutate it, the application is responsible for providing
external synchronization.
The library does protect the integrity of its own internal state (such as cached attributes and reference counting), so that concurrent reads of the same object, or any kind of concurrent use of distinct objects, cannot corrupt the interpreter. This is an integrity guarantee only: the ordering and outcome of concurrent operations on a shared object are otherwise undefined.
Additional limitations apply because cuda.core inherits the concurrency
constraints of the underlying CUDA driver. Distinct cuda.core objects can
share driver or context state, so operating on separate objects is not always
safe. For example, modifying peer device access from one thread while another
thread accesses device memory affected by that change is unsafe, even though the
two threads use different objects. The application is responsible for
synchronizing operations that concurrently read and modify shared driver state.
CUDA graph access#
Applications must serialize CUDA access to a shared graph hierarchy across host
threads, including read-only calls and calls through different views of that
hierarchy. graph.GraphDefinition.clone() accesses the source hierarchy.
graph.GraphDefinition.find_node() accesses both the source and the clone.
Hold the relevant lock or locks for the entire call. Independent graph
hierarchies can use separate locks.
See the CUDA Driver API’s Graph object thread safety documentation.