qwen_vl model family
Tasks
Supported architectures and task heads
The Hugging Face values below are copied from checkpoint metadata at the recorded revision. The TRTMC task contract comes from the exact E2E recipe. Architecture identity selects or describes a source graph; it does not imply that TRTMC reproduces every Hugging Face head with the same model_type. For example, an encoder recipe may consume the base model and intentionally return hidden states instead of the checkpoint's pretraining or classification logits.
HF model_type | HF architecture / pipeline class | TRTMC task contract | Exact recipe profiles |
|---|---|---|---|
qwen2_5_vl | Qwen2_5_VLForConditionalGenerationSource: config.architecturesMetadata: config.json at 66285546d2b8 | vision_language_generation | qwen25vl-3bqwen25vl-3b-tp2 |
qwen3_vl | Qwen3VLForConditionalGenerationSource: config.architecturesMetadata: config.json at 89644892e4d8 | vision_language_generation | qwen3-vl-2bqwen3-vl-2b-tp4 |
Declared recipes
Each row comes from a model-owned E2E manifest. It declares a test recipe; it is not a live pass receipt for every hardware target.
| Recipe | Exact Hugging Face checkpoint | Task | Build configuration | Declared E2E cases | Manifest |
|---|---|---|---|---|---|
qwen25vl-3b | Qwen/Qwen2.5-VL-3B-Instruct | fp32; single device | qwen25vl-3b | tests/e2e/models/qwen_vl/manifests/qwen25vl-3b.json | |
qwen25vl-3b-tp2 | Qwen/Qwen2.5-VL-3B-Instruct | family default; TP2 | qwen25vl-3b-tp2 | tests/e2e/models/qwen_vl/manifests/qwen25vl-3b-tp2.json | |
qwen3-vl-2b | Qwen/Qwen3-VL-2B-Instruct | bf16; single device | qwen3-vl-2b | tests/e2e/models/qwen_vl/manifests/qwen3-vl-2b.json | |
qwen3-vl-2b-tp4 | Qwen/Qwen3-VL-2B-Instruct | family default; TP4 | qwen3-vl-2b-tp4 | tests/e2e/models/qwen_vl/manifests/qwen3-vl-2b-tp4.json |
Family-owned configuration
These keys are extracted from the family's registered config schema. Pass one key per repeatable --set namespace.field=value argument. Build-time keys belong on trtmc build; session keys belong on the family's inference command.
qwen_vl_decoder
Schema source: python/tensorrt_model_connect/families/qwen_vl/runtime_config_schema.py
--set key | Type | Default | CLI surface | Allowed configuration layers |
|---|---|---|---|---|
qwen_vl_decoder.decode_attention | string | "native" | build | Build Time, Bundle Default, Session Request |
qwen_vl_decoder.max_prefill_length | int32 | 0 | build | Build Time, Bundle Default, Session Request |
qwen_vl_decoder.opt_prefill_length | int32 | 64 | build | Build Time, Bundle Default, Session Request |
qwen_vl_decoder.builder_workspace_gib | int32 | 1 | build | Build Time, Bundle Default, Session Request |
trtmc build <hf-id> --output <bundle.bundle> --set qwen_vl_decoder.decode_attention=<value>qwen_vl_lora
Schema source: python/tensorrt_model_connect/families/qwen_vl/runtime_config_schema.py
--set key | Type | Default | CLI surface | Allowed configuration layers |
|---|---|---|---|---|
qwen_vl_lora.enabled | bool | false | build | Build Time, Bundle Default, Session Request |
qwen_vl_lora.max_rank | int32 | 0 | build | Build Time, Bundle Default, Session Request |
qwen_vl_lora.target_modules | string | "q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj" | build | Build Time, Bundle Default, Session Request |
trtmc build <hf-id> --output <bundle.bundle> --set qwen_vl_lora.enabled=<value>qwen_vl_vision
Schema source: python/tensorrt_model_connect/families/qwen_vl/runtime_config_schema.py
--set key | Type | Default | CLI surface | Allowed configuration layers |
|---|---|---|---|---|
qwen_vl_vision.image_height | int32 | 448 | build | Build Time, Bundle Default, Session Request |
qwen_vl_vision.image_width | int32 | 448 | build | Build Time, Bundle Default, Session Request |
qwen_vl_vision.dynamic_resolution | bool | false | build | Build Time, Bundle Default, Session Request |
qwen_vl_vision.min_pixels | int32 | 0 | build | Build Time, Bundle Default, Session Request |
qwen_vl_vision.opt_pixels | int32 | 200704 | build | Build Time, Bundle Default, Session Request |
qwen_vl_vision.max_pixels | int32 | 0 | build | Build Time, Bundle Default, Session Request |
trtmc build <hf-id> --output <bundle.bundle> --set qwen_vl_vision.image_height=<value>The registry rejects unknown namespaces and fields. A --config file can set the same keys in JSON or YAML form.
Family-specific CLI contracts
Inputs and options below are filtered by the declared E2E task and the methods and configuration fields used by that family's native runtime implementation. The global CLI parser accepts a wider union of flags; flags absent here are not declared for this family.
trtmc run
Generate text from an image and text prompt.
Declared recipes: qwen25vl-3b, qwen25vl-3b-tp2, qwen3-vl-2b, qwen3-vl-2b-tp4
trtmc run <bundle.bundle> --prompt "<text>" --image <input.png> [generation options]| Supported input or option | Requirement | Runtime behavior |
|---|---|---|
--prompt <TEXT> | Required | Text prompt for the vision-language recipe. |
--image <PATH> | Required by declared image recipes | Image input consumed by the runtime image-aware generate overload. |
--max-new-tokens <N> | Optional | Limit the number of generated tokens or audio frames. |
--temperature <F> | Optional | Set sampling temperature. |
--top-k <N> | Optional | Restrict sampling to the top K tokens. |
--top-p <F> | Optional | Enable nucleus sampling at probability P. |
--min-p <F> | Optional | Filter tokens below min-p times the maximum probability. |
--seed <N> | Optional | Set the sampling or diffusion seed. |
--lora-adapter <DIR> --lora-adapter-id <ID> | Optional | Load and select a dynamic LoRA adapter. |
--greedy | Optional | Select deterministic greedy decoding by setting temperature to zero. |
Code basis
Runtime provider: qwen_vl
include/trtmc/pipeline.hsrc/cli/main.cppsrc/runtime/models/qwen_vl/pipeline.cppsrc/runtime/models/qwen_vl/pipeline.hsrc/runtime/models/qwen_vl/sampler.cpptests/e2e/models/qwen_vl/manifests/qwen25vl-3b-tp2.jsontests/e2e/models/qwen_vl/manifests/qwen25vl-3b.jsontests/e2e/models/qwen_vl/manifests/qwen3-vl-2b-tp4.jsontests/e2e/models/qwen_vl/manifests/qwen3-vl-2b.json
Bundle lifecycle commands
These commands apply to every declared recipe in this family; they do not add task inputs or model capabilities.
trtmc build
Build one exact checkpoint into a TensorRT-Model-Connect bundle.
trtmc build <hf-id> --output <bundle.bundle>trtmc inspect
Inspect bundle metadata, runtime identity, and packaged sections.
trtmc inspect <bundle.bundle>See the CLI Reference for all options and limitations.