model_utils

Utility functions for model type detection and classification.

MODEL_NAME_TO_TYPE={'GPT2': 'gpt', 'Mllama': 'mllama', 'Llama4': 'llama4', 'Llama': 'llama', 'Mistral': 'llama', 'GPTJ': 'gptj', 'FalconForCausalLM': 'falcon', 'RWForCausalLM': 'falcon', 'baichuan': 'baichuan', 'MPT': 'mpt', 'Bloom': 'bloom', 'ChatGLM': 'chatglm', 'Qwen3Moe': 'qwen3moe', 'Qwen3Next': 'qwen3next', 'QWen': 'qwen', 'RecurrentGemma': 'recurrentgemma', 'DiffusionGemma': 'diffusion_gemma', 'Gemma3': 'gemma3', 'Gemma2': 'gemma2', 'Gemma': 'gemma', 'phi3small': 'phi3small', 'phi3': 'phi3', 'PhiMoEForCausalLM': 'phi3', 'phi': 'phi', 'TLGv4ForCausalLM': 'phi', 'MixtralForCausalLM': 'llama', 'ArcticForCausalLM': 'llama', 'StarCoder': 'gpt', 'Dbrx': 'dbrx', 'T5': 't5', 'Bart': 'bart', 'GLM': 'glm', 'InternLM2ForCausalLM': 'internlm', 'ExaoneForCausalLM': 'exaone', 'NemotronH': 'nemotron_h', 'Nemotron': 'gpt', 'Deepseek': 'deepseek', 'Whisper': 'whisper', 'gptoss': 'gptoss', 'MiniMax': 'minimax'}

Classes

TiedWeightMap

Name-based lookups over HF's {alias: canonical} tie map (model.all_tied_weights_keys).

Functions

get_language_model_from_vl	Extract the language model lineage from a Vision-Language Model (VLM).
get_model_type	Try get the model type from the model name.
is_multimodal_model	Check if a model is a Vision-Language Model (VLM) or multimodal model.

class TiedWeightMap

Bases: object

Name-based lookups over HF’s {alias: canonical} tie map (model.all_tied_weights_keys).

Export sites ask for a group key: both sides of a tie share one key, an untied parameter returns None. The key is a name, so it survives packing / FSDP / offload, where a data_ptr would not.

__init__(model)

Source the tie map from HF’s all_tied_weights_keys (transformers >=5.0).

HF’s {target: source} == our {alias: canonical}, resolved at load, config-gated, torch.equal-pruned, and name-based so it survives FSDP shard / offload. Absent on transformers <5.0 -> empty map (the data_ptr backstop in postprocess is the net).

Parameters:

model (Module)

Return type:

None

container_group_key(container_name, first_proj_attr)

Group key for a fused-experts container, or None if untied.

The tie lives on the container’s 3-D projection (e.g. …experts.gate_up_proj); stripping that suffix gives one key shared by all the container’s projections.

Parameters:

• container_name (str)

• first_proj_attr (str)

Return type:

str | None

group_key(param_full_name)

Canonical group key for a parameter name, or None if untied.

Both sides of a tie return the same key, so it does not matter which side export visits first.

Parameters:

param_full_name (str)

Return type:

str | None

get_language_model_from_vl(model)

Extract the language model lineage from a Vision-Language Model (VLM).

This function handles the common patterns for accessing the language model component in various VLM architectures. It checks multiple possible locations where the language model might be stored.

Parameters:

model – The VLM model instance to extract the language model from

Returns:

the lineage path towards the language model

Return type:

list

Examples

>>> # For LLaVA-style models
>>> lineage = get_language_model_from_vl(vlm_model)
>>> # lineage[0] is vlm_model
>>> # lineage[1] is vlm_model.language_model

get_model_type(model)

Try get the model type from the model name. If not found, return None.

is_multimodal_model(model)

Check if a model is a Vision-Language Model (VLM) or multimodal model.

This function detects various multimodal model architectures by checking for: - Standard vision configurations (vision_config) - Language model attributes (language_model) - Nemotron-Parse conditional generation models

Parameters:

model – The HuggingFace model instance to check

Returns:

True if the model is detected as multimodal, False otherwise

Return type:

bool

Examples

>>> model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")
>>> is_multimodal_model(model)
True