Official Support Matrix#
This page lists supported software stacks. See Supported Models for checkpoint IDs and Installation for build commands.
Platforms#
Platform |
Level |
OS / SDK |
CUDA Toolkit |
TensorRT |
Build location |
Precision constraint |
|---|---|---|---|---|---|---|
Jetson Thor |
Official |
JetPack 7.0 / 7.1 |
13.0 |
JetPack package |
Device |
Model-dependent |
Jetson Thor |
Official |
JetPack 7.2 |
13.2 |
JetPack package |
Device |
Model-dependent |
NVIDIA DRIVE Thor |
Official |
DriveOS 7.2 |
13.3 |
DriveOS SDK package |
SDK container, then deploy |
Model-dependent |
NVIDIA DGX Spark (GB10) |
Official |
DGX Spark software stack |
13.0 |
System package |
Device |
Model-dependent |
Jetson Orin |
Official |
JetPack 7.2 |
13.2 |
JetPack package |
Device |
FP16, INT8, and INT4 models; FP8 KV cache |
NVIDIA GB200 workstation |
Developer |
Ubuntu 24.04 |
13.4 |
11.x |
Workstation |
Model-dependent |
NVIDIA GB300 workstation |
Developer |
Ubuntu 24.04 |
13.4 |
11.x |
Workstation |
Model-dependent |
NVIDIA IGX Thor |
Official |
Current Linux stack |
13.0 |
10.13.3.9 |
Device |
Model-dependent; SM110 iGPU or SM120 dGPU |
x86-64 Linux GPU |
Developer |
Ubuntu 22.04 / 24.04 |
12.x or 13.x |
Compatible user package |
Workstation |
Development and validation |
Official combinations are release-tested deployment targets. Compatible
combinations are expected to work with the stated constraints. Developer
combinations support development but are not edge deployment targets.
Jetson Orin does not run FP8 or FP4 model compute. Its FP8 KV-cache path converts cached values to FP16 before attention. Edge deployments normally use the TensorRT version supplied by the platform SDK; x86 builds must use mutually compatible TensorRT and CUDA packages.
The GB200 SM100 and GB300 SM103 rows are development configurations, not edge deployment targets. Their wheel payloads are cross-built; release qualification still requires installed-wheel inference on each target.
Wheel Packaging Matrix#
Release wheels use cp310-abi3 and support qualified GIL CPython 3.10 through 3.14 on x86_64 (glibc 2.35+)
and aarch64 (ARM64, glibc 2.39+). pip selects the wheel for your CPU architecture. Each wheel includes the compiled components for the matching
configurations below; you do not need a separate download for each GPU.
Your platform, CUDA and TensorRT runtime libraries, and GPU must match a listed configuration. Edge-LLM checks this when it loads. See published-wheel installation to get started.
CPU architecture |
Platform and OS / SDK |
CUDA |
GPU (SM) |
TensorRT |
|---|---|---|---|---|
x86_64 |
Ubuntu 22.04 |
13 |
80 |
10 |
x86_64 |
Ubuntu 24.04 |
13 |
86, 100, 120 |
10 |
x86_64 |
Ubuntu 24.04 |
13 |
80, 86, 100, 120 |
11 |
aarch64 |
Jetson Orin, JetPack 7.2 |
13 |
87 |
Platform-provided 10 |
aarch64 |
Jetson Thor, JetPack 7.0 / 7.1 / 7.2 |
13 |
110 |
Platform-provided 10 |
aarch64 |
DRIVE Thor, DriveOS 7.2 |
13 |
110 |
Platform-provided 10 |
aarch64 |
IGX Thor, current platform software |
13 |
110, 120 |
Platform-provided 10 |
aarch64 |
DGX Spark, current platform software |
13 |
121 |
Platform-provided 10 |
aarch64 |
GB200, Ubuntu 24.04 |
13 |
100 |
11 |
aarch64 |
GB300, Ubuntu 24.04 |
13 |
103 |
11 |
SM identifies the GPU’s compute capability: for example, SM80 means compute capability 8.0. The GPU must match a listed SM; Edge-LLM does not substitute components built for another GPU or TensorRT major version.
Use the NVIDIA driver required by your CUDA version and platform SDK.
If your configuration is not listed, follow the source installation instructions.