specs

Per-model descriptor classes.

ModelSpec is the one global descriptor of a model: everything modelopt knows about a model type lives on a single instance, resolved by config.model_type. It is composed from section mixins so each concern stays a small, separate class:

• topic sections hold architecture facts shared across subsystems (MoESpec: what a model’s MoE blocks are; NormSpec: norm-layer conventions);

• subsystem sections hold one subsystem’s per-model policy (ExportSpec; quantization / speculative-decoding sections to follow).

Sections hold per-model data plus trivial accessors over that data; subsystem logic never lives here. A model file registers exactly one ModelSpec, filling only the sections it customizes (see models/).

Classes

ExportSpec	Subsystem section: per-model policy of the unified HF export path.
MoESpec	Topic section: MoE architecture facts — the model's MoE-block layout(s).
MoEVariant	One concrete MoE-block layout of a model.
ModelSpec	The one global per-model descriptor, composed from the section mixins.
NormSpec	Topic section: normalization-layer architecture facts.

Functions

match_class_names

Return True if any of names equals a class name in module's MRO.

class ExportSpec

Bases: object

Subsystem section: per-model policy of the unified HF export path.

Architecture facts (MoE block classes, expert naming) live in MoESpec; this section holds decisions that belong to the export/quantization algorithms only.

__init__(*, pqs_fuse_rules=())

Parameters:

pqs_fuse_rules (tuple[tuple[tuple[str, ...], str, str], ...])

Return type:

None

pqs_fuse_rules: tuple[tuple[tuple[str, ...], str, str], ...] = ()

AWQ pre_quant_scale fusion rules, each a (module_class_substrings, fuse_into, fuse_from) triple: for a module whose class name contains one of the substrings, the pre_quant_scale on fuse_from is folded into fuse_into (e.g. attention o_proj -> v_proj, MLP down_proj -> up_proj). A rule is a validated mathematical-equivalence claim for that model’s modules, which is why it is declared per model rather than applied generically.

class MoESpec

Bases: object

Topic section: MoE architecture facts — the model’s MoE-block layout(s).

This describes what a model’s MoE blocks are — which class, what the expert projections are called — so any modelopt subsystem (export, quantization, speculative decoding, …) can read it instead of keeping its own per-model MoE table.

__init__(*, moe_variants=())

Parameters:

moe_variants (tuple[MoEVariant, ...])

Return type:

None

expert_linear_names_for(module)

Resolve module’s expert linear names within this model.

When every variant agrees on one naming, the module’s class is irrelevant (a spec can provide naming without the block class being known); with several namings, the module’s class picks the variant.

Return type:

tuple[str, …] | None

match_moe_variant(module)

Return the variant whose block_names matches module, else None.

Return type:

MoEVariant | None

moe_variants: tuple[MoEVariant, ...] = ()

The model’s MoE-block layouts; more than one when the same checkpoint materializes differently (see MoEVariant).

class MoEVariant

Bases: object

One concrete MoE-block layout of a model.

A model type usually has exactly one; it has several when the same checkpoint materializes with different module classes and projection names (e.g. Mixtral across transformers generations). block_names tells the variants apart.

__init__(*, block_names=(), expert_linear_names=None, has_iterable_experts=False, gate_up_pair=None)

Parameters:

• block_names (tuple[str, ...])

• expert_linear_names (tuple[str, ...] | None)

• has_iterable_experts (bool)

• gate_up_pair (tuple[str, str] | None)

Return type:

None

block_names: tuple[str, ...] = ()

MoE block class names, matched against the module’s MRO (case-insensitive exact names, not substrings; see match_class_names).

Type:

Matching key

expert_linear_names: tuple[str, ...] | None = None

Expert linear projection names, e.g. ("gate_proj", "down_proj", "up_proj"). For layouts modelopt rewrites (e.g. quantized DBRX), these are the names on the rewritten module.

gate_up_pair: tuple[str, str] | None = None

The (gate, up) pair among expert_linear_names that serving engines fuse into a single gate_up_proj, e.g. ("gate_proj", "up_proj") or ("w1", "w3"). None for non-gated experts (NemotronH) and already-fused layouts (GptOss, DBRX). Consumed by amax syncing before quantized export (see sync_moe_gate_up_amax) and by calibration grouping.

has_iterable_experts: bool = False

True when experts are per-expert iterable sub-modules (Mixtral, Qwen MoE, NemotronH, Gemma4) and can be grouped by get_experts_list; False for stacked or fused layouts (DBRX, GptOss). NOTE: currently also doubles as the grouped-export support gate, so it is conservatively False for structurally iterable but unvalidated models (see deepseek).

class ModelSpec

Bases: MoESpec, NormSpec, ExportSpec

The one global per-model descriptor, composed from the section mixins.

Resolved by HF model type (see registry.get_spec); a model registers exactly one instance, filling only the sections it customizes.

__init__(*, pqs_fuse_rules=(), weight_plus_one_norm_names=(), moe_variants=(), model_type)

Parameters:

• pqs_fuse_rules (tuple[tuple[tuple[str, ...], str, str], ...])

• weight_plus_one_norm_names (tuple[str, ...])

• moe_variants (tuple[MoEVariant, ...])

• model_type (str)

Return type:

None

model_type: str

The HF model type this spec describes (config.model_type, e.g. "qwen3_moe"). Unique across the registry.

class NormSpec

Bases: object

Topic section: normalization-layer architecture facts.

__init__(*, weight_plus_one_norm_names=())

Parameters:

weight_plus_one_norm_names (tuple[str, ...])

Return type:

None

weight_plus_one_norm_names: tuple[str, ...] = ()

Class names of norm layers whose stored weight is w - 1 (the effective scale is weight + 1), e.g. Gemma’s RMSNorm variants and LayerNorm1P. Matched against a norm module’s MRO (case-insensitive exact names). Engines must account for the +1 when folding scales into the norm weight (AWQ pre_quant_scale fusion). A structural fallback (zero_centered_gamma) stays in the engine.

match_class_names(module, names)

Return True if any of names equals a class name in module’s MRO.

Case-insensitive exact-name comparison against cls.__name__ for every class in type(module).__mro__ — the same semantics as the export dispatch registry’s string keys (modelopt.torch.export.registry). Dynamically generated quantized classes are subclasses of the original module class, so they match through their base; exact-name comparison avoids substring false positives. Comparison is case-insensitive because some registered names predate this registry and their casing was never exercised by the legacy substring matching.

Parameters:

names (tuple[str, ...])

Return type:

bool