User guide

• Overview
  • Terminology
  • Programming model
  • Requirements
    • Supported GPUs
    • Software
      • CUDA Bindings
      • Setting CUDA Installation Path
  • Missing CUDA Features
• Writing CUDA Kernels
  • Introduction
  • Kernel declaration
  • Kernel invocation
    • Choosing the block size
    • Multi-dimensional blocks and grids
  • Thread positioning
    • Absolute positions
    • Further Reading
• Memory management
  • Data transfer
    • Device arrays
  • Pinned memory
  • Mapped memory
  • Managed memory
  • Streams
  • Shared memory and thread synchronization
    • Dynamic Shared Memory
  • Local memory
  • Constant memory
  • Deallocation Behavior
    • defer_cleanup()
• Writing Device Functions
• Supported Python features in CUDA Python
  • Language
    • Execution Model
    • Floating Point Error Model
    • Constructs
    • Recursion
  • Built-in types
  • Built-in functions
  • Standard library modules
    • cmath
    • math
    • operator
  • NumPy support
  • CFFI support
• CUDA Fast Math
• Supported Atomic Operations
  • Example
• Cooperative Groups
  • Supported features
  • Using Grid Groups
  • Cooperative Launches
    • _Kernel.max_cooperative_grid_blocks()
  • Applications and Example
• Random Number Generation
  • A simple example
  • An example of managing RNG state size and using a 3D grid
• Device management
  • Device Selection
• The Device List
  • numba.cuda.cudadrv.devices.gpus
• Device UUIDs
• Examples
  • Vector Addition
  • 1D Heat Equation
  • Shared Memory Reduction
  • Dividing Click Data into Sessions
  • JIT Function CPU-GPU Compatibility
  • Monte Carlo Integration
  • Matrix multiplication
  • Calling a NumPy UFunc
• Debugging CUDA Python with the the CUDA Simulator
  • Using the simulator
  • Supported features
• GPU Reduction
  • The Reduce class
    • Reduce
      • Reduce.__init__()
      • Reduce.__call__()
• CUDA Ufuncs and Generalized Ufuncs
  • Example: Basic Example
  • Example: Calling Device Functions
  • Generalized CUDA ufuncs
• Sharing CUDA Memory
  • Sharing between process
    • Export device array to another process
      • IpcArrayHandle
        • IpcArrayHandle.close()
        • IpcArrayHandle.open()
    • Import IPC memory from another process
      • cuda.open_ipc_array()
• CUDA Array Interface (Version 3)
  • Python Interface Specification
    • Synchronization
      • Definitions
      • Design Motivations
      • Interface Requirements
      • Synchronization in Numba
      • An example launching on an array’s non-default stream
    • Lifetime management
      • Data
      • Streams
    • Lifetime management in Numba
      • Producing Arrays
      • Consuming Arrays
        • cuda.as_cuda_array()
        • cuda.from_cuda_array_interface()
    • Pointer Attributes
    • Differences with CUDA Array Interface (Version 0)
    • Differences with CUDA Array Interface (Version 1)
    • Differences with CUDA Array Interface (Version 2)
    • Interoperability
• External Memory Management (EMM) Plugin interface
  • Overview of External Memory Management
    • Effects on Deallocation Strategies
    • Management of other objects
    • Asynchronous allocation and deallocation
  • Implementing an EMM Plugin
    • BaseCUDAMemoryManager
      • BaseCUDAMemoryManager.memalloc()
      • BaseCUDAMemoryManager.memhostalloc()
      • BaseCUDAMemoryManager.mempin()
      • BaseCUDAMemoryManager.initialize()
      • BaseCUDAMemoryManager.get_ipc_handle()
      • BaseCUDAMemoryManager.get_memory_info()
      • BaseCUDAMemoryManager.reset()
      • BaseCUDAMemoryManager.defer_cleanup()
      • BaseCUDAMemoryManager.interface_version
    • The Host-Only CUDA Memory Manager
      • HostOnlyCUDAMemoryManager
        • HostOnlyCUDAMemoryManager.memhostalloc()
        • HostOnlyCUDAMemoryManager.mempin()
        • HostOnlyCUDAMemoryManager.reset()
        • HostOnlyCUDAMemoryManager.defer_cleanup()
    • The IPC Handle Mixin
      • GetIpcHandleMixin
        • GetIpcHandleMixin.get_ipc_handle()
  • Classes and structures of returned objects
    • Memory Pointers
      • MemoryPointer
      • AutoFreePointer
      • MappedMemory
      • PinnedMemory
    • Memory Info
      • MemoryInfo
        • MemoryInfo.free
        • MemoryInfo.total
    • IPC
      • IpcHandle
  • Setting the EMM Plugin
    • Environment variable
    • Function
      • set_memory_manager()
      • Resetting the memory manager
• CUDA Bindings
  • Per-Thread Default Streams
  • Roadmap
• Calling foreign functions from Python kernels
  • Device function ABI
  • Declaration in Python
    • declare_device()
  • Passing pointers
  • Linking and Calling functions
  • C/C++ Support
  • Complete Example
• Compiling Python functions for use with other languages
  • Using the C ABI
  • C and Numba ABI examples
• On-disk Kernel Caching
  • Compute capability considerations
• CUDA Minor Version Compatibility
  • Installation
    • CUDA 12
    • CUDA 11
  • Enabling MVC Support
  • References
• CUDA Frequently Asked Questions
  • nvprof reports “No kernels were profiled”