Text Generation
Build and inspect an exact text-generation checkpoint, then choose deterministic or sampled decoding.
trtmc run model.bundle \
--prompt "Summarize TensorRT in one sentence." \
--max-new-tokens 48 \
--greedy
For sampling, omit --greedy and set only the controls required by the
experiment:
trtmc run model.bundle \
--prompt "Write a two-line GPU poem." \
--max-new-tokens 64 \
--temperature 0.8 \
--top-p 0.9 \
--seed 17
| Goal | Controls |
|---|---|
| Reproducible smoke test | --greedy and a fixed prompt/output bound |
| Reproducible sampling comparison | Fixed --seed plus identical temperature/top-k/top-p/min-p |
| Chat formatting | --chat-template; inspect the packaged tokenizer/template assets |
| Suppress model thinking mode where supported | --no-thinking |
| Language-controlled seq2seq | Source/forced-BOS token IDs required by the exact family contract |
Use Sampling Reference for algorithm semantics, CLI Reference for every accepted option, and the Text Generation Tutorial for a progressive lab with exercises.