Reducing Compilation and Startup Time#

Warp loads a module’s kernels the first time one of them is used on a device. If no matching binary is cached, Warp compiles the module first. Later runs are usually faster because Warp can load the cached binary. Applications with many kernels, generic instances, or MathDx-backed tile operations may still spend noticeable time compiling.

Measuring compilation and startup time#

Start by checking the module-load output. By default, Warp prints the module name, target device, elapsed load time, and whether it compiled the module or loaded it from cache. These lines show which modules take time to load and whether each was compiled or loaded from cache.

Set warp.config.log_level to wp.LOG_DEBUG to print the start of each module load and details such as block_dim. Use this output to see which module is loading during a long startup pause.

Compilation can reuse artifacts from several independent caches:

  • Warp’s kernel cache.

  • Warp’s LTO cache for MathDx-backed tile operations.

  • The NVIDIA CUDA driver’s compute cache.

See Benchmarking Cold-Start Compilation for a cold-start benchmarking workflow and compile-time tracing when the total module-load time does not show where the delay occurs.

Reduce compilation work#

Define kernels and overloads before first module load#

Warp compiles kernels by module. Once Warp has compiled or loaded a module variant, adding new kernels or overloads changes the module hash. This includes kernels defined at runtime inside Python functions. On the next launch, Warp loads the resulting variant from cache or compiles it.

Define a module’s expected kernels and overloads before its first load so later launches can reuse the loaded module or cached artifacts.

Prefer this structure:

import warp as wp


@wp.kernel
def first_kernel(values: wp.array[float]):
    values[wp.tid()] *= 2.0


@wp.kernel
def second_kernel(values: wp.array[float]):
    values[wp.tid()] += 1.0


def run(values):
    wp.launch(first_kernel, dim=values.shape, inputs=[values])
    wp.launch(second_kernel, dim=values.shape, inputs=[values])

If a kernel must be defined later, use module="unique" to assign it to a separate module:

def make_scale_kernel(scale: float):
    @wp.kernel(module="unique")
    def scale_kernel(values: wp.array[float]):
        values[wp.tid()] *= scale

    return scale_kernel

The separate module still requires compilation, but it does not change the already loaded module. See Kernel Settings for accepted module values.

For generic kernels, declare the type combinations you expect to launch before the first launch:

from typing import Any

import warp as wp


@wp.kernel
def scale(x: wp.array[Any], s: Any):
    i = wp.tid()
    x[i] = x[i] * s


scale_f32 = wp.overload(scale, {"x": wp.array[wp.float32], "s": wp.float32})
scale_f64 = wp.overload(scale, {"x": wp.array[wp.float64], "s": wp.float64})

Only declare overloads that the application actually uses. Excessive overloads increase generated code and compile work.

Limit runtime specializations#

Warp normally chooses the CUDA target architecture from the local device, so architecture variants matter mainly when compiling modules ahead of time.

Each compiled module variant has a fixed block_dim. wp.launch() and wp.launch_tiled() select the variant that matches the launch’s block_dim. If it is not loaded, Warp loads it from cache or compiles it. When kernels in the same module use different values, Warp may compile the whole module once for each value. In that case, group kernels by block_dim in separate modules, or use module="unique" for a kernel that needs its own value.

When preloading modules, include only the variants the application expects to use. Each unused block_dim or tile combination adds compile work.

Set compilation options before importing or loading modules#

Module options affect compiled output and cache reuse. Configure global compilation options before importing modules that define kernels. Changing them later can cause modules to use different settings or require additional compilation.

Set module-specific options before the module’s first load. Changing them after a launch or explicit load can trigger another compilation and reduce cache reuse. Avoid changing the same module’s options repeatedly during a run.

Disable backward code generation when gradients are not needed#

Warp generates backward code for kernels that may participate in automatic differentiation. If an application does not need gradients for those kernels, disable backward code generation to reduce generated code and compile work.

Disable it globally before module creation:

import warp as wp

wp.config.enable_backward = False

For modules that do not need gradients, set the module option before the module loads:

import warp as wp
import my_app.kernels

wp.set_module_options({"enable_backward": False}, module=my_app.kernels)

For individual kernels, use the decorator option:

@wp.kernel(enable_backward=False)
def integrate(values: wp.array[float]):
    values[wp.tid()] += 1.0

Only disable backward generation for kernels whose adjoints will not be used by Tape or differentiable framework integrations.

Choose module boundaries#

Put kernels that are deployed, loaded, and changed together in the same module. This can avoid repeating fixed code-generation, compilation, and load costs across several small modules.

Use separate modules for kernels that change independently, are rarely used, require different compilation options, or use different CUDA block_dim values. A broad module can compile more code when its contents or options change. Many tiny modules can duplicate shared functions and structs and add fixed load-time overhead. Runtime-created unique modules also cannot always be preloaded by package name.

Use module="unique" to isolate one kernel in a separate module. Add module_options=... when that kernel also needs different compilation options:

@wp.kernel(module="unique", module_options={"enable_backward": False})
def halve_values(values: wp.array[float]):
    i = wp.tid()
    values[i] *= 0.5

Compile known kernels before latency-sensitive work#

Preloading and ahead-of-time compilation work only for kernels and variants that already exist. An application that creates a kernel or specialization from runtime data must wait for that data before Warp can compile or load it. Create known variants during the build or before latency-sensitive work begins.

Load independent modules in parallel#

When compilation work is spread across several independent modules, wp.load_module() can compile or load them concurrently:

import warp as wp
import my_app.kernels

wp.load_module(
    my_app.kernels,
    device="cuda:0",
    recursive=True,
    max_workers=4,
)

The submodules must already be imported and registered with Warp. Parallel loading offers little benefit when one large module dominates or when module loads are short.

Without multiple workers, wp.load_module() runs the same compilation earlier, before CUDA graph capture or a frame loop.

Compile known modules for deployment#

For an application with a known set of kernels and a controlled deployment environment, use wp.compile_aot_module() during the build and load its output with wp.load_aot_module() at runtime. This example builds a CUBIN for a deployment GPU with compute capability 9.0:

# build_kernels.py
import warp as wp
import my_app.kernels

wp.compile_aot_module(
    my_app.kernels,
    arch=90,  # Deployment GPU has compute capability 9.0
    module_dir="build/warp_modules",
    use_ptx=False,
)

Deploy build/warp_modules with the application to a machine with that GPU architecture, then load the module at runtime:

# run_app.py
import warp as wp
import my_app.kernels

wp.load_aot_module(my_app.kernels, module_dir="build/warp_modules", use_ptx=False)

Choose the output format based on the deployment environment:

  • CUBIN avoids driver JIT compilation. Build a CUBIN for each GPU architecture the application loads. For a known GPU target, CUBIN may also work with an older driver under CUDA minor-version compatibility when PTX produced by the same newer Toolkit would not. Features that require a newer driver remain unavailable.

  • PTX lets the driver compile for compatible GPU architectures when the module loads. The deployed driver must support the PTX emitted by the CUDA Toolkit used to build Warp. Without a CUDA forward-compatibility package, PTX produced by a newer Toolkit does not work on an older driver.

See Compatibility & Support for CUDA driver and GPU architecture compatibility.

Development iteration tradeoffs#

These settings can shorten development compile times, but may reduce runtime performance or available features. Measure their effect on the target workload.

Lower optimization levels#

A lower optimization level may shorten compile time while iterating. For CUDA, this setting affects compilation only when the Warp native library was built with CUDA Toolkit 12.9 or newer:

import warp as wp

wp.config.optimization_level = 0

Restore the application’s normal optimization level before benchmarking runtime performance.

Use the native tile-matmul fallback#

MathDx compilation for wp.tile_matmul() can dominate cold compile time, especially when backward code generation adds adjoint GEMMs. Different tile dimensions or dtypes can require additional MathDx LTO compilation. If it slows development, try the Warp/native GEMM fallback temporarily:

import warp as wp

wp.config.enable_mathdx_gemm = False

Use the module option when only one module should use the fallback:

import my_app.kernels

wp.set_module_options({"enable_mathdx_gemm": False}, module=my_app.kernels)

Other MathDx-backed tile operations, including solvers and FFTs, may also incur LTO compilation.