Limitations and Known Issues#
This page documents the known limitations and issues for each release version of TensorRT Edge-LLM.
0.12.0#
On Windows with RTX GPUs (TensorRT-RTX 1.7), engine builds for some large NVFP4 MoE and dense models (27B-35B class) can fail with a TensorRT internal error in
safeMemcpyHostToDevice(Error Code 2). The fix is planned for the next version of TensorRT-RTX.Windows on ARM is not supported. The Windows source build in Installation targets x64 only.
On Jetson Thor, prefill throughput for some models, including native, EAGLE, MTP, and DFlash execution with NVFP4 or FP8 weights, is lower than in 0.10.1. The 0.11 token-major MLP layout prevents TensorRT from fusing the gate and up projections into a dual GEMM. A fix is planned for a later release.
NVFP4 quantization of models with a Conv3d vision tower, such as Qwen3-Omni-30B-A3B-Instruct and Alpamayo-R1-10B, fails during calibration with
Error building extension 'conv3d_implicit_gemm_cuda_v20_wmma'. ModelOpt 0.45.0 compiles this extension as C++17, but the PyTorch 2.14.0 headers require C++20. Downgrade to PyTorch 2.13.0 in the quantization environment, with the matching torchvision. Other NVFP4 VLMs, and FP16 or INT4-AWQ checkpoints of these models, are not affected.The published NVFP4 (W4A16) Nemotron-3.5-Bolt-E4B DFlash draft has a low acceptance rate. Set
EDGELLM_DFLASH_DENSE_LAST_LAYER=1for the ONNX export and fortrtedge buildto run the draft’s last layer as dense FP16. The variable applies only to a Qwen3-style DFlash draft behind a Gemma4 target.
0.10.0#
Fixed in 0.10.1:
Int4GroupwiseGemmPluginV2may cause accuracy degradation. For ONNX export, use--int4-gemm-plugin-version 1to select the V1 plugin as a fallback.
0.9.0#
Some FP8 ViT configurations may produce NaN outputs for specific models. If this occurs, use FP16 ViT for the visual encoder.
0.7.0#
Fixed in 0.7.1:
tensorrt-edgellm-quantizeexposed FP8 visual-tower quantization for supported VLM checkpoints, but other visual-tower precisions were not exposed in 0.7.0.Fixed in 0.7.1: Qwen3-TTS required a hybrid export workflow in 0.7.0.
Reduced-vocabulary exports with packed INT4 LM heads require
group_size=128and a reduced vocabulary size that is a multiple of 128.
0.6.1#
Some large models (7B-8B) may encounter cudaMallocAsync issue on DriveOS. Please increase huge page by
echo 15658 | sudo tee /proc/sys/vm/nr_hugepages
0.6.0#
TensorRT 10.15 may cause accuracy degradation with NVFP4 for some models. Use TensorRT 10.13.3.9 shipped with Jetpack 7.1 instead.
Fixed in 0.7.0: Setting
--max-batch-size=1could cause accuracy degradation or engine build failure for some models. Setting--max-batch-size=2was the 0.6.0 workaround.Fixed in 0.7.0: CuTe DSL kernels could hang for some inputs when runtime batch size > 1, so the kernels were only enabled when runtime batch size = 1 in 0.6.0.