config#

Saved execution policy for GDN/KDA state QAT.

Classes

`LinearAttentionConfig`	One execution policy shared by the chunked prefix and recurrent suffix.
`LinearAttentionPolicyEntry`	Assign a complete policy to supported modules matching `module_name`.

class LinearAttentionConfig#

Bases: ModeloptBaseConfig

One execution policy shared by the chunked prefix and recurrent suffix.

serving uses native forward arithmetic with a differentiable adjoint; fla is the disabled/default path. vllm targets vLLM 0.30.0’s Triton prefill and single-token, non-speculative decode with BF16 operands and FP32 recurrent storage. Deployment must explicitly select Triton prefill (gdn_prefill_backend="triton") and enable packed recurrent decode (VLLM_ENABLE_FLA_PACKED_RECURRENT_DECODE=1). Mixed prefill/decode batches and automatic FlashInfer/CuTe dispatch are outside this profile. KDA uses the standalone FLA/Triton API, with no model-level serving guarantee. vllm_kimi_k3 explicitly selects vLLM’s Kimi-K3 Triton kernels (also used by Kimi-Linear in vLLM 0.30), raw projections and that model package’s dependencies. These targets are saved in ModelOpt/Megatron policy metadata, not HF export; they do not configure or validate a remote server. Kimi serving must set both kda_prefill_backend and kda_decode_backend to triton; the supported gate is unbounded softplus. replayssm targets the quantized-ReplaySSM fork’s INT8/Hadamard kernels. One quantized checkpoint feeds both reads and recurrence updates; separate read and recurrence precisions require the GDN-only replayssm_two_precision profile: an FP32 anchor, FP8 block128 read copy along K, and a fixed 16-token window. It imports upstream ReplaySSM emulation; runtime caches use FP32 storage. For the shared-checkpoint fork, replay_window=1 refreshes every token. Fresh prefill has no internal state QDQ. Two-precision replay reads the unrounded prefill checkpoint for its first output, then encodes the read copy.

TensorQuantizer enables QDQ and owns the public profiles’ format and grouping; the ReplaySSM fork owns its checkpoint encoding. state_block_v controls grouping only for legacy per-tile quantizers.

backend: Literal['fla', 'serving']#

model_config = {'extra': 'forbid', 'validate_assignment': True}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

precision: Literal['vllm', 'vllm_kimi_k3', 'replayssm', 'replayssm_two_precision']#

replay_window: int#

state_block_v: Literal[16, 32, 64, 128]#

property state_codec#: The native profile fixes the codec; it is not an independent setting.

class LinearAttentionPolicyEntry#

Bases: ModeloptBaseConfig

Assign a complete policy to supported modules matching module_name.

Rules apply in order: the last match wins, without merging fields. Unmatched rules are ignored locally, as with quant_cfg, so a shared recipe can be applied independently to pipeline stages and virtual-pipeline chunks.

cfg: LinearAttentionConfig#

model_config = {'extra': 'forbid', 'validate_assignment': True}#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

module_name: str#