Examples

New to Edge-LLM? Follow Quick Start to start a chat or launch an HTTP server, then explore the examples below.

Choose the workflow by its input and output contract. The Supported Models page maps each model family to these contracts.

Use the trtedge CLI for chat, serving, engine building, ONNX export, and model preparation.

Text Output

• Interactive Text Chat: local streaming conversations with trtedge chat

• Python runtime: build a text engine and run inference from local model files

• Phi-4 Multimodal: merge the vision LoRA adapter before quantization and export

• Speculative Decoding: MTP, EAGLE3, DFlash, DSpark, and JetSpec acceleration

• ASR: audio to transcript text with Qwen3-ASR

• Nemotron-3.5-ASR: audio to transcript text with a model-specific RNN-T runtime

Speech Output

• Qwen3-TTS: text, style, language, or reference speech to 24 kHz speech

• Qwen3-Omni: text, images, or audio to text and optional 24 kHz speech

Action Output

• Vision-Language-Action: Alpamayo trajectory generation, and Cosmos3 and pi0.5 policy action generation

Serving and Evaluation

• Experimental Python API and Server: Python and OpenAI-compatible interfaces

• NeMo Evaluator: evaluate a local OpenAI-compatible server