Build from Source#

Building from source is mostly intended for developers who wish to modify, customize, and contribute to TensorRT LLM. If you only need to run TensorRT LLM, use the Installation Guide instead.

Prerequisites#

Use Docker to build and run TensorRT LLM. Instructions to install an environment to run Docker containers for the NVIDIA platform can be found here.

TensorRT LLM uses git-lfs, which needs to be installed in advance:

sudo apt-get update && sudo apt-get -y install git git-lfs
git lfs install

Step 1: Clone the Repository#

git clone https://github.com/NVIDIA/TensorRT-LLM.git
cd TensorRT-LLM
git lfs pull

Step 2: Pull the Development Container#

Pull the pre-built TensorRT LLM devel container from NGC. Replace x.y.z with the desired version. Browse the available tags on NGC to find the latest release.

docker pull nvcr.io/nvidia/tensorrt-llm/devel:1.3.0rc25

Step 3: Start the Container#

From the repository root, start a development container with the source tree mounted into it.

docker run --rm -it \
        --ipc=host \
        --ulimit memlock=-1 --ulimit stack=67108864 \
        --gpus=all \
        --volume <path_to_tensorrt_llm_on_host>:<path_to_tensorrt_llm_in_container> \
        --workdir <path_to_tensorrt_llm_in_container> \
        nvcr.io/nvidia/tensorrt-llm/devel:1.3.0rc25

Note on Docker flags

--ulimit memlock=-1 allows unlimited locked memory, which is needed for GPU workloads.
--ulimit stack=67108864 sets the stack size to 64 MB to prevent stack overflows in deeply nested C++/CUDA code paths.

Step 4: Build TensorRT LLM#

Once inside the container, build TensorRT LLM from source using scripts/build_wheel.py. Run python3 ./scripts/build_wheel.py --help for the full list of options.

Typical development build#

Build the C++ code, skip wheel packaging, and use symlinks so that changes are reflected immediately. Then install in editable mode for Python development.

python3 scripts/build_wheel.py --use_ccache -a "90-real" --skip_building_wheel --linking_install_binary
pip install -e .

Key flags used above:

Flag	Purpose
`--use_ccache`	Use ccache for faster incremental rebuilds
`-a "90-real"`	Build only for a specific GPU architecture (e.g. Hopper). Reduces compile time significantly. See support-matrix-hardware for values.
`--skip_building_wheel`	Skip `.whl` packaging – only needed for distribution, not development
`--linking_install_binary`	Symlink built libraries instead of copying them
`pip install -e .`	Editable install so Python changes take effect without reinstalling

Other common options#

Flag	Purpose
`--clean`	Clean the build directory before building
`--build_type RelWithDebInfo`	Build with debug info (default: `Release`)
`-j <N>`	Number of parallel compile jobs (default: number of available CPUs)
`--fast_build`	Skip compiling some kernels to speed up compilation – for development only
`--cpp_only`	Build only the C++ runtime library, without Python bindings

Building from a checkout on a network filesystem#

Network filesystems (Lustre, NFS, GPFS) handle large streaming I/O well but are slow for metadata-heavy workloads. A full build creates a very large number of small files (CMake state, object files, downloaded dependencies, the build virtual environment, wheel staging), so keeping that state in the checkout makes builds on such filesystems far slower than necessary.

Pass --build_root (or set the TRTLLM_BUILD_ROOT environment variable) to keep all high-churn build state on fast local storage while the checkout stays on shared storage:

python3 scripts/build_wheel.py --build_root /tmp/trtllm-build --use_ccache -a "90-real" --skip_building_wheel --linking_install_binary

With --build_root <dir> set, the following default under <dir> instead of the checkout:

State	Location under `<dir>`	Individual override
CMake build directory (objects, `_deps` downloads, conan output)	`cpp-build*`	`--build_dir`
Build virtual environment	`venv-<python version>`	run inside an activated venv, or `--no-venv`
Wheel staging tree and `*.egg-info`	`wheel-staging`	`TRTLLM_WHEEL_STAGING_DIR`
ccache directory (with `--use_ccache`)	`ccache`	`CCACHE_DIR`
Intermediate extension-module objects	`kv_cache_manager_v2-temp`	—

Conan’s cpp/CMakeUserPresets.json convenience file is also skipped in this mode, since it would reference the (possibly ephemeral) out-of-tree build directory.

Only final artifacts are still written into the checkout: tensorrt_llm/libs, tensorrt_llm/include, Python bindings and stubs, generated FMHA kernel sources, the configured cpp/include/tensorrt_llm/executor/version.h, and the .whl output directory (--dist_dir).

Related knobs for shared-storage workflows:

CCACHE_DIR: point at persistent storage so compile results survive container or job restarts even when <dir> is ephemeral (for example, node-local /tmp).
CONAN_HOME: conan’s download cache defaults to ~/.conan2; relocate it if your home directory is small or slow.
--use-3rdparty-cache: cache FetchContent git clones as bare repos under TRTLLM_FETCHCONTENT_CACHE (defaults to 3rdparty/.cache_3rdparty), avoiding repeated full clones after a clean.

Plain local-disk builds are unaffected: without --build_root, all paths behave as before.

Artifact copy-back into the checkout (tensorrt_llm/include, the deep_gemm/deep_ep/flash_mla Python trees) is incremental: directory trees are populated once via a streamed tar pipeline and then kept in sync by size/mtime comparison, so a rebuild rewrites only what changed instead of re-copying ~10k files onto the (possibly network) filesystem.

Out-of-tree wheel builds (read-only checkout)#

For CI or ephemeral-node workflows that only need a wheel, add --out-of-tree to guarantee the checkout is never written — it can even be mounted read-only:

python3 scripts/build_wheel.py --build_root /tmp/trtllm-build --out-of-tree --use_ccache -a "90-real"

In this mode the generated FMHA kernel sources and the configured version.h go to the build tree (via the TRTLLM_FMHA_GEN_DIR and TRTLLM_VERSION_H_INCLUDE_DIR CMake variables), and the wheel is assembled from a staging copy of the Python package under <build_root>/package, landing in <build_root>/dist by default. Submodules and git-lfs content must be materialized before the build (the usual git submodule update --init --recursive), since the build will not modify the checkout. Editable-install workflows (--skip_building_wheel, --linking_install_binary, --install) are incompatible with --out-of-tree: they import compiled artifacts from the checkout by design. --version-override is also incompatible, since it edits tensorrt_llm/version.py in the checkout.

Python-only build (no C++ compilation)#

If you only need to modify Python code, you can skip C++ compilation entirely by reusing precompiled binaries:

TRTLLM_USE_PRECOMPILED=1 pip install -e .

This downloads a precompiled wheel matching the version in tensorrt_llm/version.py and extracts its compiled libraries into your working directory. Override the version with TRTLLM_USE_PRECOMPILED=x.y.z or specify a custom URL/path with TRTLLM_PRECOMPILED_LOCATION.