Examples#
New to Edge-LLM? Follow Quick Start to start a chat or launch an HTTP server, then explore the examples below.
Choose the workflow by its input and output contract. The Supported Models page maps each model family to these contracts.
Use the trtedge CLI for chat, serving, engine building, ONNX export, and model preparation.
Text Output#
Interactive Text Chat: local streaming conversations with
trtedge chatText generation: one shared workflow for dense LLM, MoE, VLM, and Omni text output; Phi-4 Multimodal adds the model-specific vision-LoRA merge
Python runtime: build a text engine and run inference from local model files
Speculative Decoding: MTP, EAGLE3, DFlash, DSpark, and JetSpec acceleration
ASR: audio to transcript text with Qwen3-ASR
Nemotron-3.5-ASR: audio to transcript text with a model-specific RNN-T runtime
Speech Output#
Qwen3-TTS: text, style, language, or reference speech to 24 kHz speech
Qwen3-Omni: text, images, or audio to text and optional 24 kHz speech
Video Output#
MiniMax-H3: text to video with stereo audio from the FP16 checkpoint
Action Output#
Vision-Language-Action: Alpamayo trajectory generation, and Cosmos3 and pi0.5 policy action generation
Use with other software#
Integration: AIPerf, NeMo Evaluator, VLMEvalKit, Claude Code, OpenClaw, and Codex
Python API and Server: Python and OpenAI-compatible interfaces