Supported Models

Supported checkpoint IDs are listed below. See the support matrix for platform coverage and the examples for workflows.

Support Policy

TensorRT Edge-LLM supports the checkpoint IDs listed below. Dense LLM families include official dense checkpoints below 30B parameters. Larger dense checkpoints and non-dense variants require case-by-case validation. MoE, multimodal, audio, TTS, omni, EAGLE3, DFlash, and JetSpec support is limited to the listed rows.

The model coverage list is not comprehensive, and not every listed checkpoint has been fully verified on every supported platform and precision. If a listed model does not export, build, or run correctly, please report an issue with the checkpoint ID, precision, platform, and command line used.

The model class names were checked against the upstream Transformers model source tree. Checkpoint IDs are linked to their Hugging Face pages and grouped into original checkpoints and quantized checkpoints.

Precision Notes

• Dense precision set: FP16/BF16 checkpoints, ModelOpt FP8/MXFP8/FP4/NVFP4/INT4 AWQ/INT8 SmoothQuant checkpoints, and INT4 GPTQ checkpoints. INT8 GPTQ is not supported.

• Jetson Orin supports FP16, INT8, and INT4 runtime precision in the supported JetPack configurations. Do not select FP8, MXFP8, FP4, or NVFP4 checkpoints for Orin.

• For INT4 engine builds on Jetson Orin devices with less system memory, such as Jetson Orin Nano, pass --externalize-weights int4_ffn for dense checkpoints or --externalize-weights int4_ffn int4_moe for MoE checkpoints to reduce engine build memory.

• For FP16/BF16 source checkpoints, use the Quantization script to create a unified quantized checkpoint for tensorrt_edgellm, then export the generated checkpoint.

• FP8 KV cache is detected automatically from checkpoint metadata by tensorrt_edgellm.

• tensorrt-edgellm-export exports visual encoders. Use tensorrt-edgellm-quantize llm --visual_quantization fp8 before export when FP8 visual weights are required.

• MXFP8 and FP4/NVFP4 require Blackwell-class hardware for runtime execution.

• For platform-specific NVFP4 MoE layouts, follow Export NVFP4 MoE for SM12x before building the engine.

Text Generation

Llama 3.x

Original:

• meta-llama/Meta-Llama-3-8B-Instruct

• meta-llama/Llama-3.1-8B-Instruct

• meta-llama/Llama-3.2-1B-Instruct

• meta-llama/Llama-3.2-3B-Instruct

Pre-quantized:

• nvidia/Llama-3.1-8B-Instruct-FP8

• nvidia/Llama-3.1-8B-Instruct-NVFP4

Qwen2 / Qwen2.5

Original:

• Qwen/Qwen2-0.5B, Qwen/Qwen2-0.5B-Instruct

• Qwen/Qwen2-1.5B, Qwen/Qwen2-1.5B-Instruct

• Qwen/Qwen2-7B, Qwen/Qwen2-7B-Instruct

• Qwen/Qwen2-Math-1.5B, Qwen/Qwen2-Math-1.5B-Instruct

• Qwen/Qwen2-Math-7B, Qwen/Qwen2-Math-7B-Instruct

• Qwen/Qwen2.5-0.5B, Qwen/Qwen2.5-0.5B-Instruct

• Qwen/Qwen2.5-1.5B, Qwen/Qwen2.5-1.5B-Instruct

• Qwen/Qwen2.5-3B, Qwen/Qwen2.5-3B-Instruct

• Qwen/Qwen2.5-7B, Qwen/Qwen2.5-7B-Instruct

• Qwen/Qwen2.5-14B, Qwen/Qwen2.5-14B-Instruct

• Qwen/Qwen2.5-Coder-0.5B, Qwen/Qwen2.5-Coder-0.5B-Instruct

• Qwen/Qwen2.5-Coder-1.5B, Qwen/Qwen2.5-Coder-1.5B-Instruct

• Qwen/Qwen2.5-Coder-3B, Qwen/Qwen2.5-Coder-3B-Instruct

• Qwen/Qwen2.5-Coder-7B, Qwen/Qwen2.5-Coder-7B-Instruct

• Qwen/Qwen2.5-Coder-14B, Qwen/Qwen2.5-Coder-14B-Instruct

• Qwen/Qwen2.5-Math-1.5B, Qwen/Qwen2.5-Math-1.5B-Instruct

• Qwen/Qwen2.5-Math-7B, Qwen/Qwen2.5-Math-7B-Instruct

• deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B, deepseek-ai/DeepSeek-R1-Distill-Qwen-7B, deepseek-ai/DeepSeek-R1-Distill-Qwen-14B

Pre-quantized:

• Qwen/Qwen2-0.5B-Instruct-AWQ, Qwen/Qwen2-0.5B-Instruct-GPTQ-Int4

• Qwen/Qwen2-1.5B-Instruct-AWQ, Qwen/Qwen2-1.5B-Instruct-GPTQ-Int4

• Qwen/Qwen2-7B-Instruct-AWQ, Qwen/Qwen2-7B-Instruct-GPTQ-Int4

• Qwen/Qwen2.5-0.5B-Instruct-AWQ, Qwen/Qwen2.5-0.5B-Instruct-GPTQ-Int4

• Qwen/Qwen2.5-1.5B-Instruct-AWQ, Qwen/Qwen2.5-1.5B-Instruct-GPTQ-Int4

• Qwen/Qwen2.5-3B-Instruct-AWQ, Qwen/Qwen2.5-3B-Instruct-GPTQ-Int4

• Qwen/Qwen2.5-7B-Instruct-AWQ, Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4

• Qwen/Qwen2.5-14B-Instruct-AWQ, Qwen/Qwen2.5-14B-Instruct-GPTQ-Int4

• Qwen/Qwen2.5-Coder-0.5B-Instruct-AWQ, Qwen/Qwen2.5-Coder-0.5B-Instruct-GPTQ-Int4

• Qwen/Qwen2.5-Coder-1.5B-Instruct-AWQ, Qwen/Qwen2.5-Coder-1.5B-Instruct-GPTQ-Int4

• Qwen/Qwen2.5-Coder-3B-Instruct-AWQ, Qwen/Qwen2.5-Coder-3B-Instruct-GPTQ-Int4

• Qwen/Qwen2.5-Coder-7B-Instruct-AWQ, Qwen/Qwen2.5-Coder-7B-Instruct-GPTQ-Int4

• Qwen/Qwen2.5-Coder-14B-Instruct-AWQ, Qwen/Qwen2.5-Coder-14B-Instruct-GPTQ-Int4

Qwen3

Original:

• Qwen/Qwen3-0.6B, Qwen/Qwen3-0.6B-Base

• Qwen/Qwen3-1.7B, Qwen/Qwen3-1.7B-Base

• Qwen/Qwen3-4B, Qwen/Qwen3-4B-Base

• Qwen/Qwen3-4B-Instruct-2507, Qwen/Qwen3-4B-Thinking-2507

• Qwen/Qwen3-8B, Qwen/Qwen3-8B-Base

• Qwen/Qwen3-14B, Qwen/Qwen3-14B-Base

Pre-quantized:

• Qwen/Qwen3-4B-AWQ

• Qwen/Qwen3-8B-AWQ, nvidia/Qwen3-8B-FP8, nvidia/Qwen3-8B-NVFP4

• Qwen/Qwen3-14B-AWQ, nvidia/Qwen3-14B-FP8, nvidia/Qwen3-14B-NVFP4

• Qwen/Qwen3-30B-A3B-GPTQ-Int4, nvidia/Qwen3-30B-A3B-NVFP4

Qwen vision-language families

Qwen2.5-VL:

• Qwen/Qwen2.5-VL-3B-Instruct, Qwen/Qwen2.5-VL-7B-Instruct

• Qwen/Qwen2.5-VL-3B-Instruct-AWQ, Qwen/Qwen2.5-VL-7B-Instruct-AWQ

• nvidia/Qwen2.5-VL-7B-Instruct-FP8, nvidia/Qwen2.5-VL-7B-Instruct-NVFP4

Qwen3-VL and compatible checkpoints:

• Qwen/Qwen3-VL-2B-Instruct, Qwen/Qwen3-VL-2B-Thinking

• Qwen/Qwen3-VL-4B-Instruct, Qwen/Qwen3-VL-4B-Thinking

• Qwen/Qwen3-VL-8B-Instruct, Qwen/Qwen3-VL-8B-Thinking

• nvidia/Cosmos-Reason2-2B, nvidia/Cosmos-Reason2-8B

Qwen3.5 / Qwen3.6 / Qwen3.8

• Qwen/Qwen3.5-0.8B, Qwen/Qwen3.5-0.8B-Base

• Qwen/Qwen3.5-2B, Qwen/Qwen3.5-2B-Base

• Qwen/Qwen3.5-4B, Qwen/Qwen3.5-4B-Base

• Qwen/Qwen3.5-9B, Qwen/Qwen3.5-9B-Base

• Qwen/Qwen3.5-27B

• Qwen/Qwen3.6-27B

• Qwen/Qwen3.8-27B

• RadixArk/Qwen3.8-27B-NVFP4

• Qwen/Qwen3.5-35B-A3B-GPTQ-Int4, nvidia/Qwen3.6-35B-A3B-NVFP4

InternVL and Phi-4

InternVL:

• OpenGVLab/InternVL3-1B-hf, OpenGVLab/InternVL3-2B-hf

• OpenGVLab/InternVL3-8B-hf

• OpenGVLab/InternVL3-14B-hf

• OpenGVLab/InternVL3_5-1B-HF, OpenGVLab/InternVL3_5-2B-HF, OpenGVLab/InternVL3_5-4B-HF

• OpenGVLab/InternVL3_5-8B-HF, OpenGVLab/InternVL3_5-14B-HF

• OpenGVLab/InternVL3-1B-AWQ, OpenGVLab/InternVL3-2B-AWQ, OpenGVLab/InternVL3-8B-AWQ, OpenGVLab/InternVL3-14B-AWQ

Phi-4:

• microsoft/Phi-4-multimodal-instruct

Nemotron-3 / 3.5

• nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16

• nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8

• nvidia/NVIDIA-Nemotron-Nano-9B-v2

• nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8, nvidia/NVIDIA-Nemotron-Nano-9B-v2-NVFP4

• nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4

• nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4

• nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4

• nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-NVFP4

Gemma4

Text, image, and audio input:

• google/gemma-4-E2B-it, google/gemma-4-E4B-it

• google/gemma-4-12B-it

Text and image input:

• google/gemma-4-31B-it, nvidia/Gemma-4-31B-IT-NVFP4

• google/gemma-4-26B-A4B-it-qat-q4_0-unquantized

• nvidia/Gemma-4-26B-A4B-NVFP4

MTP assistants:

• google/gemma-4-E2B-it-assistant, google/gemma-4-E4B-it-assistant

• google/gemma-4-12B-it-assistant, google/gemma-4-31B-it-assistant

• google/gemma-4-26B-A4B-it-assistant

DiffusionGemma

• nvidia/diffusiongemma-26B-A4B-it-NVFP4

Speech Recognition

• Qwen3-ASR: Qwen/Qwen3-ASR-0.6B, Qwen/Qwen3-ASR-1.7B

• Nemotron-3.5-ASR: nvidia/nemotron-3.5-asr-streaming-0.6b

Speech Generation

• Qwen3-TTS: Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice, Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice, Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign, Qwen/Qwen3-TTS-12Hz-0.6B-Base, Qwen/Qwen3-TTS-12Hz-1.7B-Base

• Qwen3-Omni: Qwen/Qwen3-Omni-30B-A3B-Instruct

Action and Multimodal Reasoning

• Alpamayo: nvidia/Alpamayo-R1-10B

• Cosmos3-Edge: nvidia/Cosmos3-Edge, nvidia/Cosmos3-Edge-Policy-DROID

Speculative Draft Checkpoints

EAGLE3 Draft Models

Draft checkpoint	Base model
yuhuili/EAGLE3-LLaMA3.1-Instruct-8B	meta-llama/Llama-3.1-8B-Instruct
AngelSlim/Qwen3-1.7B_eagle3	Qwen/Qwen3-1.7B
AngelSlim/Qwen3-4B_eagle3	Qwen/Qwen3-4B
Tengyunw/qwen3_8b_eagle3 or AngelSlim/Qwen3-8B_eagle3	Qwen/Qwen3-8B
Rayzl/qwen2.5-vl-7b-eagle3-sgl	Qwen/Qwen2.5-VL-7B-Instruct

DFlash Draft Models

Draft checkpoint	Base model
z-lab/Qwen3-4B-DFlash-b16	Qwen/Qwen3-4B-Instruct-2507
z-lab/Qwen3-8B-DFlash-b16	Qwen/Qwen3-8B
z-lab/Qwen3.5-4B-DFlash	Qwen/Qwen3.5-4B
z-lab/Qwen3.5-4B-DFlash (quantized checkpoint: Qwen3.5-4B-DFlash-NVFP4)	Qwen3.5-4B-NVFP4
z-lab/Qwen3.5-9B-DFlash	Qwen/Qwen3.5-9B
z-lab/Qwen3.5-27B-DFlash	Qwen/Qwen3.5-27B
z-lab/Qwen3.5-35B-A3B-DFlash	Qwen/Qwen3.5-35B-A3B-GPTQ-Int4
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DFlash	nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4

DSpark Draft Models

Draft checkpoint	Base model
deepseek-ai/dspark_qwen3_4b_block7	Qwen/Qwen3-4B
deepseek-ai/dspark_qwen3_8b_block7	Qwen/Qwen3-8B
deepseek-ai/dspark_gemma4_12b_block7	google/gemma-4-12B-it
RadixArk/Qwen3.8-27B-DSpark	RadixArk/Qwen3.8-27B-NVFP4

JetSpec Draft Models

JetSpec draft checkpoints are detected by jetspec_config in config.json and exported with DFlashDraftModel using causal proposal attention. The validated runtime path is branching tree verification: export the base with --jetspec-tree-base --jetspec-draft-dir <draft_checkpoint>, export the draft with --jetspec-draft --jetspec-draft-dir <draft_checkpoint>, and run with --specDraftTopK > 1. --jetspecBlockSize and --dflashBlockSize configure the same cached-draft proposal block size; the JetSpec spelling is provided for clarity in JetSpec command lines.

For the listed Qwen3 pair, disable thinking mode in the input JSON when evaluating accuracy, acceptance rate, or throughput.

Draft checkpoint	Base model
JetSpec/jetspec-qwen3-8b	Qwen/Qwen3-8B