Model Types

namespace trt_edgellm

Argument bundles for the CuTe DSL FMHA launchers.

Each public run* entry point fills one of these once from its own arguments and members, and the launcher then spreads it across the generated descriptors. One struct per wrapper shape rather than a single superset: a superset would let a field that matters on one path (kvCacheCapacity on the dense path, tokensPerPage on the paged one) sit silently at zero on another.

These carry no generated types, so this header is safe to include from translation units that use the required FMHA-v2 runner, the optional CUTE_DSL_FMHA_BLACKWELL_ENABLED runner, or both. The descriptor-filling machinery itself lives in cuteDslTensorDescriptors.h, which stays free of any FMHA concept.

Helpers for populating the tensor descriptors emitted by the CuTe DSL C exporter.

Every AOT variant exports its own nominally distinct but layout-identical descriptor structs (fmha_d64_Tensor_q_tensor_t vs fmha_d128_Tensor_q_tensor_t) plus a cute_dsl_<variant>_wrapper entry point. There is no umbrella C type, so descriptor types are recovered here from the signature of the wrapper that consumes them: a call site names only the wrapper and the kernel module, and pairing a descriptor with the wrong variant is not expressible.

The exporter (cutlass/cute/export/c_header_generator.py) always names the members data, dynamic_shapes and dynamic_strides, but emits each array only when its dynamic mask is non-empty. A rank-1 descriptor therefore has no dynamic_strides member at all and needs makeCuSeqLenTensor() rather than the strided builders below.

namespace multimodal

Enums

enum class ModelType

Enum for supported multimodal model types.

Values:

enumerator QWEN2_VL

Qwen2-VL model.

enumerator QWEN2_5_VL

Qwen2.5-VL model.

enumerator QWEN3_VL

Qwen3-VL model.

enumerator QWEN3_5

Qwen3.5 model.

enumerator QWEN3_OMNI_AUDIO_ENCODER

Qwen3-Omni audio encoder (3-stage CNN, 4x downsample)

enumerator QWEN3_OMNI_NEXT_AUDIO_ENCODER

Qwen3-Next Omni audio encoder (4-stage CNN, 8x downsample)

enumerator QWEN3_OMNI_VISION_ENCODER

Qwen3-Omni / Qwen3-Next Omni vision encoder (image-to-embeddings)

enumerator QWEN3_OMNI_CODE2WAV

Qwen3-Omni Code2Wav vocoder (codes-to-waveform)

enumerator INTERNVL

InternVL model.

enumerator PHI4MM

Phi-4MM model.

enumerator GEMMA4_VISION

Gemma4 vision encoder.

enumerator GEMMA4_UNIFIED_VISION

Encoder-free Gemma4 Unified vision embedder.

enumerator GEMMA4_UNIFIED_AUDIO

Encoder-free Gemma4 Unified audio embedder.

enumerator NEMOTRON_OMNI_VISION_ENCODER

Nemotron-Omni vision encoder.

enumerator NEMOTRON_OMNI_AUDIO_ENCODER

Nemotron-Omni audio encoder.

enumerator NEMOTRON3_5_ASR_AUDIO_ENCODER

Nemotron-3.5-ASR FastConformer encoder (RNN-T)

enumerator GEMMA4_AUDIO_ENCODER

Gemma4 audio encoder.

enumerator COSMOS3_EDGE

Cosmos3-Edge reasoner vision encoder (SigLIP2 + PatchMerger)

enumerator UNKNOWN

Unknown or unsupported model type.

Functions

inline ModelType stringToModelType(std::string const &modelTypeStr)

Convert string to ModelType enum

Parameters:

modelTypeStr – String representation of model type

Returns:

Corresponding ModelType enum value