Migration Guide: TensorRT Backend Removed#

Note

Breaking change. The TensorRT engine backend has been removed. PyTorch is now the sole execution backend for TensorRT LLM (AutoDeploy, built on the PyTorch backend, remains available).

What changed#

Removed

Replacement / new behavior

LLM(backend="tensorrt")

Raises ValueError — PyTorch is the only backend; omit backend

TrtLlmArgs

Use TorchLlmArgs (the default)

tensorrt_llm._tensorrt_engine.LLM

Use tensorrt_llm.LLM

trtllm-build / trtllm-refit / trtllm-prune

No engine-build step — HuggingFace checkpoints load directly

Per-model convert_checkpoint.py

Not needed — no checkpoint conversion

--backend tensorrt (CLI)

Omit, or pass --backend pytorch

tensorrt pip dependency

Dropped (no longer installed)

How to migrate#

Python API — PyTorch is the default, so drop the backend argument:

from tensorrt_llm import LLM

llm = LLM(model="<hf_model>")

Command line — --backend pytorch is the default, so no engine is needed:

trtllm-serve <hf_model>

There is no separate checkpoint-conversion or engine-build step.

Examples#

PyTorch usage lives with the LLM API examples (for example quickstart_advanced.py, quickstart_multimodal.py) and the model-specific deployment guides. The per-model convert_checkpoint.py / trtllm-build example directories have been removed.

Retained reference docs#

The legacy tree under docs/source/legacy/ is kept for cross-reference only. Pages that are purely about the TensorRT engine mechanism carry a caution banner; the rest document concepts still relevant to the PyTorch backend.

Where to go next#