C++ and CUDA Workflows#

Warp is primarily authored from Python, but several workflows expose generated C++/CUDA code or replay captured Warp work from a native host application. This page collects the public entry points for those non-Python integration paths and links to the detailed workflow documentation and examples.

Use this page when you need to:

  • insert native C++/CUDA snippets into generated Warp kernels.

  • ahead-of-time compile Warp kernels into source, PTX, or CUBIN files.

  • load generated Warp binaries or source from a CUDA C++ application.

  • serialize captured Warp work and replay it from C++ without a Python runtime.

Native Snippets in Warp Kernels#

Use @wp.func_native to insert native C++/CUDA code into generated Warp modules. Native functions are useful when Warp does not provide a built-in operation, CUDA intrinsic, synchronization pattern, or low-level expression that your kernel needs.

Pure C++ snippets, meaning snippets without CUDA-only constructs, can be used by CPU kernels. The same snippet can also be used by CUDA kernels if the code is valid device code. CUDA-specific constructs, such as __shared__ memory, __syncthreads(), and CUDA atomics, require CUDA kernels.

The decorator takes native source code as a string. The decorated Python function is a typed stub: its arguments define the names and types available to the snippet, and its body should be ... because Warp replaces the body with the native snippet during code generation.

The thread index should be computed by the caller and passed explicitly. Native snippets are inserted into generated C++/CUDA, so they cannot call wp.tid() directly.

import numpy as np
import warp as wp

snippet = "out[tid] = x[tid] + 1.0f;"


@wp.func_native(snippet)
def increment(x: wp.array[wp.float32], out: wp.array[wp.float32], tid: int):
    ...


@wp.kernel
def increment_kernel(x: wp.array[wp.float32], out: wp.array[wp.float32]):
    tid = wp.tid()
    increment(x, out, tid)


device = "cpu"
x = wp.array(np.arange(4, dtype=np.float32), dtype=wp.float32, device=device)
out = wp.zeros_like(x)
wp.launch(increment_kernel, dim=x.shape, inputs=[x], outputs=[out], device=device)

CUDA Shared Memory#

Native snippets can use CUDA features that Warp does not expose directly. The following example performs a reduction within a single 128-thread block using shared memory. It assumes the launch uses exactly one block. Generalizing this pattern to multiple blocks requires using a per-block thread index and storing one result per block.

import numpy as np
import warp as wp

snippet = """
    __shared__ int sum[128];

    sum[tid] = arr[tid];
    __syncthreads();

    for (int stride = 64; stride > 0; stride >>= 1) {
        if (tid < stride) {
            sum[tid] += sum[tid + stride];
        }
        __syncthreads();
    }

    if (tid == 0) {
        out[0] = sum[0];
    }
    """


@wp.func_native(snippet)
def reduce(arr: wp.array[int], out: wp.array[int], tid: int):
    ...


@wp.kernel
def reduce_kernel(arr: wp.array[int], out: wp.array[int]):
    tid = wp.tid()
    reduce(arr, out, tid)


arr = wp.array(np.arange(128, dtype=np.int32), dtype=wp.int32, device="cuda")
out = wp.zeros(1, dtype=wp.int32, device="cuda")
wp.launch(reduce_kernel, dim=128, inputs=[arr], outputs=[out], block_dim=128, device="cuda")

Thread Block Clusters and Distributed Shared Memory#

CUDA Thread Block Clusters group adjacent CTAs into a cluster whose blocks the hardware guarantees to co-schedule on a single GPU Processing Cluster (GPC) — a hardware group of SMs linked by an on-chip interconnect. That SM-to-SM path unlocks distributed shared memory: each block can address the shared memory of every other block in its cluster. Clusters require compute capability 9.0 (Hopper) or higher.

Set the cluster size with the cluster_dim kernel argument. Warp emits the kernel’s __cluster_dims__ attribute from that value, but the cluster machinery itself — distributed shared memory, cluster barriers, and cluster rank queries — is reachable only from native CUDA code, so a @wp.func_native snippet is how you use it.

The example below reduces an array with a single cluster of CLUSTER_DIM blocks. Each block sums its own slice into shared memory; after a cluster barrier, block 0 reaches into every peer block’s shared memory through the __cluster_map_shared_rank device builtin to form the cluster-wide total. A second cluster barrier keeps every block alive until block 0 has finished reading, because a block must not exit while its shared memory is still being accessed.

import numpy as np
import warp as wp

CLUSTER_DIM = 4
BLOCK_DIM = 32

snippet = r"""
    const unsigned int lane = threadIdx.x;

    // Each block reduces its slice into a single shared-memory accumulator.
    __shared__ int s_partial;
    if (lane == 0) s_partial = 0;
    __syncthreads();
    atomicAdd(&s_partial, value);
    __syncthreads();

    // Cluster barrier: every block's s_partial is finalized and visible
    // across the cluster's distributed shared memory.
    asm volatile("barrier.cluster.arrive;" ::: "memory");
    asm volatile("barrier.cluster.wait;" ::: "memory");

    unsigned int rank, num_blocks;
    asm volatile("mov.u32 %0, %%cluster_ctarank;" : "=r"(rank));
    asm volatile("mov.u32 %0, %%cluster_nctarank;" : "=r"(num_blocks));

    // Block 0 gathers every peer's partial sum via distributed shared memory.
    if (rank == 0 && lane == 0) {
        int total = 0;
        for (unsigned int r = 0; r < num_blocks; ++r) {
            int *remote = (int *)__cluster_map_shared_rank(&s_partial, r);
            total += *remote;
        }
        out[0] = total;
    }

    // Second barrier: no block exits (freeing its shared memory) until
    // block 0 has finished reading every peer's accumulator.
    asm volatile("barrier.cluster.arrive;" ::: "memory");
    asm volatile("barrier.cluster.wait;" ::: "memory");
    """


@wp.func_native(snippet)
def cluster_reduce(value: int, out: wp.array[int]):
    ...


@wp.kernel(cluster_dim=CLUSTER_DIM, enable_backward=False)
def cluster_reduce_kernel(values: wp.array[int], out: wp.array[int]):
    tid = wp.tid()
    cluster_reduce(values[tid], out)


# One cluster of CLUSTER_DIM blocks: launch exactly CLUSTER_DIM * BLOCK_DIM threads.
n = CLUSTER_DIM * BLOCK_DIM
values = wp.array(np.arange(n, dtype=np.int32), dtype=wp.int32, device="cuda")
out = wp.zeros(1, dtype=wp.int32, device="cuda")
wp.launch(
    cluster_reduce_kernel,
    dim=n,
    inputs=[values],
    outputs=[out],
    block_dim=BLOCK_DIM,
    device="cuda",
)

print(int(out.numpy()[0]))
8128

The "memory" clobber on each barrier stops the compiler from reordering shared memory accesses across it. cluster_dim values 2–8 are portable across all cluster-capable devices; values 9–16 are non-portable and depend on the GPU, so query warp.get_cuda_max_cluster_dim() before using them.

Inline PTX#

Native snippets can also use inline Parallel Thread Execution (PTX) assembly inside CUDA code. Inline PTX is useful when you need a GPU instruction that is not exposed directly through Warp or CUDA C++.

The following example computes the sum of four byte-wise absolute differences between two packed 8-bit values. The PTX vabsdiff4 instruction performs four byte-wise absolute differences and, with the .add modifier, accumulates them into one 32-bit result.

import numpy as np
import warp as wp

snippet = r"""
    unsigned int result;
    unsigned int zero = 0;
    asm("vabsdiff4.u32.u32.u32.add %0, %1, %2, %3;"
        : "=r"(result)
        : "r"(a), "r"(b), "r"(zero));
    return result;
    """


@wp.func_native(snippet)
def absdiff4_sum_u8(a: wp.uint32, b: wp.uint32) -> wp.uint32:
    ...


@wp.kernel
def absdiff4_kernel(
    a: wp.array[wp.uint32],
    b: wp.array[wp.uint32],
    out: wp.array[wp.uint32],
):
    tid = wp.tid()
    out[tid] = absdiff4_sum_u8(a[tid], b[tid])


def pack4(values):
    return np.uint32(values[0] | (values[1] << 8) | (values[2] << 16) | (values[3] << 24))


a_host = np.array([pack4([10, 20, 30, 40]), pack4([0, 128, 255, 13])], dtype=np.uint32)
b_host = np.array([pack4([13, 18, 41, 35]), pack4([255, 120, 0, 15])], dtype=np.uint32)

a = wp.array(a_host, dtype=wp.uint32, device="cuda")
b = wp.array(b_host, dtype=wp.uint32, device="cuda")
out = wp.zeros_like(a)
wp.launch(absdiff4_kernel, dim=a.shape, inputs=[a, b], outputs=[out], device="cuda")

# [3 + 2 + 11 + 5, 255 + 8 + 255 + 2]
print(out.numpy().tolist())
np.testing.assert_array_equal(out.numpy(), np.array([21, 520], dtype=np.uint32))
[21, 520]

The "r" constraints bind the operands to 32-bit integer registers, which matches the .u32 instruction operands. The final PTX operand is an accumulator and is supplied as a zero-initialized register in this example. If the assembly reads or writes memory through pointers, add the appropriate "memory" clobber as described in NVIDIA’s inline PTX documentation.

Returning Values#

A native snippet can return a value when the Python stub declares a return type. Warp supports scalar, vector, matrix, quaternion, array, and fixed-array return types. Struct return values are not supported.

snippet = """
    float sq = x * x;
    return sq;
    """


@wp.func_native(snippet)
def square(x: wp.float32) -> wp.float32:
    ...

Pass-by-reference Parameters#

Native functions can use wp.ref[T] parameters for scalar, vector, matrix, quaternion, or struct values that should be mutated in place. Inside the native snippet, a wp.ref[T] parameter is visible as a C++/CUDA reference named after the Python parameter:

@wp.func_native("x = x + 5;")
def add_five(x: wp.ref[wp.int32]):
    ...


@wp.kernel(enable_backward=False)
def add_five_kernel(values: wp.array[wp.int32]):
    i = wp.tid()
    add_five(values[i])

Call sites must pass an addressable expression, such as a local variable, function parameter, array element, struct field, vector or matrix component, or nested field rooted at an array element. If the native function is used in a tape-recorded computation, provide an adj_snippet; adjoint variables for wp.ref parameters use the same adj_ prefix and are also snippet-visible as references.

Use wp.address_of(expr) when a native snippet needs a raw pointer to a specific addressable expression instead of a wp.ref[T] parameter. Use array.ptr for the base pointer of an entire array, and use wp.address_of(expr) for local variables, array elements, components, or nested fields such as wp.address_of(v.y) and wp.address_of(outers[i].inner.value):

@wp.func_native("*(float*)ptr += delta;")
def add_to_ptr(ptr: wp.uint64, delta: wp.float32):
    ...


@wp.kernel(enable_backward=False)
def add_to_ptr_kernel(values: wp.array[wp.float32]):
    i = wp.tid()
    local = wp.float32(1.0)
    add_to_ptr(wp.address_of(local), wp.float32(2.0))
    add_to_ptr(wp.address_of(values[i]), local)

Raw pointer writes performed through native snippets are not automatically differentiable by Warp. Keep those kernels forward-only or provide the appropriate manual adjoint for the surrounding native operation.

Differentiable Native Functions#

When a native function participates in a tape-recorded computation, provide an adj_snippet that accumulates adjoints for the native operation. Adjoint variables use the adj_ prefix, and return-value adjoints are named adj_ret.

snippet = "out[tid] = 2.0f * x[tid] + y[tid];"
adj_snippet = """
    adj_x[tid] += 2.0f * adj_out[tid];
    adj_y[tid] += adj_out[tid];
    """


@wp.func_native(snippet=snippet, adj_snippet=adj_snippet)
def axpy(
    x: wp.array[wp.float32],
    y: wp.array[wp.float32],
    out: wp.array[wp.float32],
    tid: int,
):
    ...

During the backward pass, Warp runs a forward replay phase. By default, native functions replay the original snippet. If the forward snippet has side effects that should not be repeated, such as mutating a counter with an atomic operation, provide replay_snippet. An empty string is a valid no-op replay snippet.

snippet = """
    int next_index = atomicAdd(counter, 1);
    thread_values[tid] = next_index;
    """
replay_snippet = ""


@wp.func_native(snippet, replay_snippet=replay_snippet)
def record_index(counter: wp.array[int], thread_values: wp.array[int], tid: int):
    ...

Native Function Limitations#

  • Native snippets are inserted into generated C++/CUDA and are not parsed as Warp code.

  • The snippet can refer to variables named after the typed Python stub arguments.

  • CUDA-specific snippets cannot run on CPU devices.

  • Type hints must accurately describe the stub arguments and return type.

  • Struct return values are unsupported.

  • Users are responsible for native-code correctness, synchronization, memory safety, and portability.

External Build Inputs#

Experimental

The external-compilation APIs described in this section are experimental and may change without deprecation in future releases. The registration functions live in warp.experimental.build, which must be imported explicitly.

Extra build inputs for a module can be grouped in wp.ModuleBuildOptions:

build_options = wp.ModuleBuildOptions(
    extra_cuda_include_dirs=[my_cuda_include_dir],
    extra_cpu_include_dirs=[my_cpu_include_dir],
    extra_cuda_preamble='#include "my_cuda_addon.h"',
    extra_cpu_preamble='#include "my_cpu_addon.h"',
    extra_build_dependencies=[my_cuda_addon_header, my_cpu_addon_header],
)
wp.set_module_options({"extra_build_options": build_options})

The preambles are inserted after Warp’s own headers, but before codegen-only cast macros and the generated code. An external header may therefore use public Warp macros such as CUDA_CALLABLE and ordinary C++ function-style casts, and the generated kernels see everything the preamble declares. A preamble cannot define macros that Warp’s headers consume, because those are already included.

Warp hashes the configured include-directory paths, but not the contents of headers resolved through them. List external headers and other build inputs in extra_build_dependencies so Warp also hashes their contents. Dependency contents are re-read whenever the module hash is recomputed — after wp.set_module_options() or in a new process — not on every launch.

Changes to a wp.ModuleBuildOptions instance take effect when it is passed to wp.set_module_options(). After modifying an instance, call wp.set_module_options() again so Warp invalidates the module’s cached compilation state.

If more than one addon contributes build inputs, combine them without discarding earlier settings:

current = wp.get_module_options(kernel.module)["extra_build_options"]
if current is None:
    current = wp.ModuleBuildOptions()
wp.set_module_options(
    {"extra_build_options": current.merged(addon_build_options)},
    module=kernel.module,
)

External Native Value Types#

External C++ value types can be used directly as kernel annotations, built-in arguments and return values, Warp struct fields, and array dtypes. Define the host ABI with ctypes.Structure and register the same class:

import ctypes
import warp as wp
import warp.experimental.build

class Color(ctypes.Structure):
    _fields_ = [
        ("r", ctypes.c_float),
        ("g", ctypes.c_float),
        ("b", ctypes.c_float),
    ]

wp.experimental.build.add_native_type(
    Color,
    native_name="render::Color",
    fields={"r": wp.float32, "g": wp.float32, "b": wp.float32},
    initializer="aggregate",
)

The exact C++ definition must be made visible using the module preamble and include-directory options above. Warp emits compile-time checks for standard layout, trivial copyability, size, and alignment. CPU and CUDA compilation also check each exposed member’s C++ type. CPU compilation checks every exposed field offset, while CUDA compilation checks each field size. Setting initializer="aggregate" opts into ordered field construction such as Color(r, g, b) inside kernels. In this mode, fields must contain every ctypes field in declaration order. Without it, only default construction is available.

Pass fields=None (the default) for an opaque type. Opaque values can cross kernel and registered built-in boundaries and can be stored in arrays, but their members and captured constants are not exposed. Warp does not manage resources referenced by opaque values; the external package remains responsible for their ownership and lifetime.

Native value types do not automatically gain arithmetic or differentiation. Register operations explicitly with warp.experimental.build.add_builtin(), and leave requires_grad disabled on Warp arrays containing native value types.

A native value can provide context to a Warp function. For example, this helper uses a native Color without differentiating the color itself:

@wp.func
def scale_with_context(color: Color, value: float):
    """Scale a differentiable value using a C++ Color."""
    return color.r * value

Warp computes the derivative with respect to value; color.r has no gradient. This also works with Warp arrays of Color values and native value fields inside Warp structs. Ordinary floating-point fields in those Warp structs remain differentiable.

External Function Signatures#

Use warp.experimental.build.add_builtin() to give an external C++ function a Python name, ordered argument types, and a return type. Omit the return type for C++ void. The returned handle supports positional and keyword calls.

Use wp.ref[T] for a mutable C++ T& argument. Given this header declaration:

namespace my_addon {
CUDA_CALLABLE inline void write(float& output, float value) {
    output = value;
}
}

Register it in Python:

write = wp.experimental.build.add_builtin(
    "my_addon_write",
    {"output": wp.ref[wp.float32], "value": wp.float32},
    native_name="my_addon::write",
)

Inside a kernel, pass an array element or a mutable local:

write(output[i], value)
local = float(0.0)
write(local, value)

These calls update the original storage. If two reference arguments point to the same element, they see each other’s writes. A literal or temporary expression, such as write(0.0, value), is rejected. Calls with mutable reference parameters require enable_backward=False unless protected by custom gradient and replay functions, as shown below.

Overloads must differ in their ordered argument types. Changing only parameter names, return types, or reference qualifiers cannot distinguish a call and is rejected. Repeating an identical registration is allowed; changing an existing registration’s derivative settings is a conflict.

External Function Gradients#

Warp does not infer derivatives from C++ headers. Choose one of these approaches:

  • Supply a C++ adjoint and register the operation with is_differentiable=True. The example below uses a float input and return value.

  • Wrap the operation in warp.func() and supply warp.func_grad() to write the derivative in Warp. This also lets you pass native value types while differentiating ordinary Warp arguments.

By default, external registrations are non-differentiable: no gradient flows through the call. The call can still run during backward replay. If it changes state, follow the replay guidance below.

Supply a C++ adjoint#

Save the following header as external_framework.h. It supplies the forward operation and its derivative:

#pragma once

namespace external_framework {

CUDA_CALLABLE inline float square(float x) {
    return x * x;
}

CUDA_CALLABLE inline void adj_square(float x, float& adj_x, float adj_result) {
    adj_x += 2.0f * x * adj_result;
}

}

Save this Python script as external_framework_example.py beside the header. Start by registering the C++ operation. Warp finds its derivative by adding adj_ to the function name in the same C++ namespace:

from pathlib import Path

import warp as wp
import warp.experimental.build

square = wp.experimental.build.add_builtin(
    "external_framework_square",
    input_types={"x": wp.float32},
    value_type=wp.float32,
    native_name="external_framework::square",
    is_differentiable=True,
)

Call it from a kernel:

@wp.kernel
def compute_loss(x: wp.array[wp.float32], loss: wp.array[wp.float32]):
    """Compute a scalar loss with the external square operation."""
    loss[0] = square(x[0])

Include the header in this kernel’s module:

header = Path(__file__).resolve().with_name("external_framework.h")
build_options = wp.ModuleBuildOptions(
    extra_cpu_include_dirs=[str(header.parent)],
    extra_cuda_include_dirs=[str(header.parent)],
    extra_cpu_preamble='#include "external_framework.h"',
    extra_cuda_preamble='#include "external_framework.h"',
    extra_build_dependencies=[str(header)],
)
wp.set_module_options({"extra_build_options": build_options}, module=compute_loss.module)

Record a launch and run backward. Append each of these blocks to the same script:

device = "cpu"  # Use "cuda:0" to run on a GPU.
x = wp.array([3.0], dtype=wp.float32, device=device, requires_grad=True)
loss = wp.zeros(1, dtype=wp.float32, device=device, requires_grad=True)

with wp.Tape() as tape:
    wp.launch(compute_loss, dim=1, inputs=[x], outputs=[loss], device=device)

tape.backward(loss=loss)

Run uv run python -i external_framework_example.py, then inspect the forward result and gradient in the same Python session:

>>> print(loss.numpy())
[9.]
>>> print(x.grad.numpy())
[6.]

tape.backward() calls external_framework::adj_square automatically. Its arguments are the original inputs, references to their gradients, and the return-value gradient (omitted for a void return). Use += to accumulate into input gradients. Here the loss gradient is one, giving dx = 2 * 3 = 6. CUDA_CALLABLE makes the same header usable on CPU and CUDA.

Use the forward result in an adjoint#

Some C++ adjoints use the forward result. For example, the derivative of sqrt(x) is 1 / (2 * result). Register such a function with both is_differentiable=True and adjoint_uses_forward_output=True:

CUDA_CALLABLE inline float root(float x) {
    return wp::sqrt(x);
}

CUDA_CALLABLE inline void adj_root(float x, float result, float& adj_x, float adj_result) {
    adj_x += adj_result / (2.0f * result);
}
root = wp.experimental.build.add_builtin(
    "external_framework_root",
    {"x": wp.float32},
    wp.float32,
    native_name="external_framework::root",
    is_differentiable=True,
    adjoint_uses_forward_output=True,
)

Place these definitions in the same external_framework namespace as the previous example. Warp recomputes root(x) during backward replay and passes that result after the original inputs, before the gradient arguments. It does not save the original forward result. The same rule applies to warp.grad(). This option requires a differentiable registration with a non-void return.

Provide a Warp gradient and safe replay#

Use warp.func_grad() to define a wrapper’s derivative in Warp. The wrapper can accept native value types, such as Color, and differentiate ordinary Warp arguments, such as float. The native value itself has no gradient.

For a wrapper that calls an external reference operation, also provide warp.func_replay(). Backward execution may need to recompute forward values; repeating the original mutation could change data twice.

Using write from above, define the forward operation, its derivative, and a replay that recomputes the result without the reference write:

@wp.func
def square_with_native_write(x: float):
    """Square a value through an external reference write."""
    result = float(0.0)
    write(result, x * x)
    return result

@wp.func_grad(square_with_native_write)
def square_grad(x: float, adj_ret: float):
    """Accumulate the square gradient."""
    wp.adjoint[x] += 2.0 * x * adj_ret

@wp.func_replay(square_with_native_write)
def square_replay(x: float):
    """Recompute the square without repeating the forward reference write."""
    return x * x

Put these callbacks on the function that contains the complete mutating operation. A callback on an inner function does not protect the ordinary forward body of an outer wrapper. A direct warp.grad() call uses the custom gradient without replaying the original function.

Your wrapper must preserve any values needed to compute gradients and avoid unintended writes during replay. Custom callbacks may call external operations, but passing an array or a native value containing a pointer by value does not make the referenced data immutable. Warp rejects unprotected wp.ref replay; it cannot detect hidden writes through a C++ pointer.

Limits when differentiating native value types#

  • Registered C++ adjoints support ordinary Warp types. Mutable reference inputs, native value inputs or outputs, and Warp arrays containing native values require a warp.func_grad() wrapper instead.

  • A Warp function returning a native value can still propagate gradients through writes to ordinary Warp arrays. The native return value has no gradient.

  • A function returning both native values and ordinary Warp values requires a custom warp.func_grad(). For warp.grad(), expose ordinary Warp outputs through a wrapper; native outputs cannot receive gradient seeds.

  • A native snippet that takes or returns native value types needs an explicit C++ adjoint to participate in backward execution. This includes native values nested inside Warp structs or arrays, and snippets that return nothing but write to differentiable arrays. Use a no-op adjoint only when no derivative contribution is needed.

Complete Addon Example#

An addon should register its types and built-ins when imported, then expose a small helper that applies its build inputs to the module containing the user’s kernels. For example, my_addon.py can contain:

import ctypes
from pathlib import Path

import warp as wp
import warp.experimental.build

include_dir = Path(__file__).parent / "include"
header = include_dir / "my_addon.h"


class Color(ctypes.Structure):
    _fields_ = [
        ("r", ctypes.c_float),
        ("g", ctypes.c_float),
        ("b", ctypes.c_float),
    ]


wp.experimental.build.add_native_type(
    Color,
    native_name="my_addon::Color",
    fields={"r": wp.float32, "g": wp.float32, "b": wp.float32},
    initializer="aggregate",
)
wp.experimental.build.add_builtin(
    "my_addon_scale_color",
    {"value": Color, "factor": wp.float32},
    Color,
    native_name="my_addon::scale_color",
)

_build_options = wp.ModuleBuildOptions(
    extra_cuda_include_dirs=[include_dir],
    extra_cpu_include_dirs=[include_dir],
    extra_cuda_preamble='#include "my_addon.h"',
    extra_cpu_preamble='#include "my_addon.h"',
    extra_build_dependencies=[header],
)


def configure_module(module):
    current = wp.get_module_options(module)["extra_build_options"]
    if current is None:
        current = wp.ModuleBuildOptions()
    wp.set_module_options(
        {"extra_build_options": current.merged(_build_options)},
        module=module,
    )

Users import the addon before compiling kernels that call its built-ins, and configure each kernel module before its first launch or AOT compilation:

import warp as wp

import my_addon


@wp.kernel
def scale_colors(colors: wp.array[my_addon.Color], factor: float):
    tid = wp.tid()
    colors[tid] = wp.my_addon_scale_color(colors[tid], factor)


my_addon.configure_module(scale_colors.module)

# A normal wp.launch() now JIT-compiles with the addon's build inputs.
# For external runtimes, AOT compilation returns the generated artifact.
artifacts = wp.compile_aot_module(
    scale_colors.module,
    device="cuda",
    use_ptx=True,
)

Registered built-in names share Warp’s global kernel namespace. Addons should use a package-specific prefix unless they intentionally add an overload to an existing Warp operation. Registrations are process-global and equivalent registrations are idempotent. Recreating a ctypes class on module reload does not make instances of the old and new Python classes interchangeable.

Ahead-of-Time C++/CUDA Workflows#

Warp can compile kernels ahead of time and write the generated CUDA source, metadata, PTX, or CUBIN files to disk. This is useful when a Python build step authors and validates kernels, but a native CUDA C++ application owns runtime execution.

The full AOT workflow is documented in Ahead-of-Time Compilation Workflows. The C++ examples under warp/examples/cpp/ show two deployment patterns:

  • 00_cubin_launch compiles a Warp kernel to a CUBIN, loads that module with the CUDA Driver API, and launches the generated kernel with cuLaunchKernel().

  • 01_source_include includes the generated .cu source in a CUDA C++ translation unit and launches the generated forward and backward kernels directly.

Both examples use warp/native/aot.h for Warp’s generated type definitions, CUDA setup helpers, and error-checking macros. The generated code also depends on the native type headers such as builtin.h that ship in warp/native/.

API Capture Replay from C++#

API Capture (APIC) can serialize a captured Warp graph to a .wrp file plus a companion module directory. The saved graph can later be loaded from Python or from a standalone C++ program that links against the Warp native library.

See API Capture: Saving and Loading Graphs for the C API surface, serialization format notes, and current limitations. The C++ examples cover both device families:

  • 02_apic_visualization records and saves a CUDA workload in Python. The C++ loader reconstructs a CUDA graph, updates named inputs, and replays the frame with cudaGraphLaunch().

  • 03_apic_visualization_cpu captures and replays on the CPU device. The C++ viewer does not link against CUDA. It loads recorded CPU kernel objects and replays the graph with wp_apic_cpu_replay_graph().

Native Library Headers#

The C++ integration examples intentionally use a small native surface:

  • warp/native/aot.h exposes utilities for generated AOT kernels and includes Warp’s generated type support.

  • warp/native/warp.h declares the core Warp C API exported by the native library.

  • warp/native/warp_clang.h declares the experimental CPU block runtime binding, module loading, and symbol lookup API exported by warp-clang.

  • warp/native/apic.h declares the APIC graph loading and replay API used by .wrp graph consumers.

Other files in warp/native/ implement Warp’s runtime and kernel support library. They are useful when inspecting generated code, but the examples above are the recommended starting points for native host integration.

Native Library Linking and Version Checks#

Windows wheels include warp.lib beside warp.dll and warp-clang.lib beside warp-clang.dll. At build time, link against the .lib import libraries and deploy the corresponding DLLs. This records load-time dependencies on the DLLs. On Linux and macOS, link the application against the Warp and warp-clang shared libraries at build time.

Warp does not currently provide an official find_package() configuration. A consuming CMake project that knows the wheel’s package location can define ordinary imported targets. For example, the Windows targets can be defined as:

set(WARP_PACKAGE_ROOT ".../site-packages/warp")

add_library(Warp::runtime SHARED IMPORTED)
set_target_properties(Warp::runtime PROPERTIES
    IMPORTED_LOCATION "${WARP_PACKAGE_ROOT}/bin/warp.dll"
    IMPORTED_IMPLIB "${WARP_PACKAGE_ROOT}/bin/warp.lib"
    INTERFACE_INCLUDE_DIRECTORIES "${WARP_PACKAGE_ROOT}/native"
)

add_library(Warp::clang SHARED IMPORTED)
set_target_properties(Warp::clang PROPERTIES
    IMPORTED_LOCATION "${WARP_PACKAGE_ROOT}/bin/warp-clang.dll"
    IMPORTED_IMPLIB "${WARP_PACKAGE_ROOT}/bin/warp-clang.lib"
    INTERFACE_INCLUDE_DIRECTORIES "${WARP_PACKAGE_ROOT}/native"
)

target_link_libraries(my_app PRIVATE Warp::runtime Warp::clang)

On Linux and macOS, set IMPORTED_LOCATION to the corresponding .so or .dylib file and omit IMPORTED_IMPLIB.

The native C APIs and .wrp format do not currently guarantee backward or forward compatibility across Warp releases. Build and deploy a native application with the headers, DLLs or other shared libraries, Windows import libraries, .wrp files, and companion module artifacts from the same Warp release. Mixing these artifacts across releases is unsupported, even when a particular combination happens to work. Check both linked libraries during startup before loading a graph or CPU module:

#include "warp.h"
#include "warp_clang.h"

#include <cstring>

if (wp_init(WP_VERSION_STRING) != 0)
    return 1;
if (std::strcmp(wp_warp_clang_version(), WP_VERSION_STRING) != 0)
    return 2;

Applications that load CPU object modules through wp_load_obj() must also bind the core Warp CPU block runtime after initialization and before loading the first object:

if (!wp_llvm_set_cpu_block_runtime(wp_cpu_block_runtime_get_api()))
    return 3;

These checks detect a library that reports a different Warp release, but they do not establish a stable ABI:

  • The Windows loader may reject a missing DLL or symbol before main(), so runtime version checks cannot recover from that failure.

  • An unchanged C symbol with a changed signature remains unsafe despite the version checks.

  • Put the matching DLLs beside the executable or configure the DLL search path deliberately.

  • Linking warp-clang.lib makes warp-clang.dll a startup dependency, even for a process that later replays only CUDA graphs.

  • Both DLLs are included in the same Warp Windows distribution package, such as a wheel or conda package, but remain separate libraries.

  • CPU APIC callers should include warp_clang.h for the runtime binding, module loading, and symbol lookup declarations instead of copying declarations from clang.cpp.