config

Configuration classes for sparse attention optimization.

Classes

`CalibrationConfig`	Configuration for automatic threshold calibration using RULER dataset.
`FlashSkipSoftmaxConfig`	Configuration for Flash Attention-aware softmax skip sparse attention.
`SparseAttentionAttributeConfig`	Sparse attention attribute configuration for pattern-based module config.
`SparseAttentionConfig`	Base configuration for sparse attention optimization.
`VSAAttributeConfig`	Video Sparse Attention (VSA) attribute configuration.
`VSAConfig`	Configuration for Video Sparse Attention optimization.

class CalibrationConfig

Bases: ModeloptBaseConfig

Configuration for automatic threshold calibration using RULER dataset.

Calibration fits an Exponential model to determine dynamic thresholds that achieve target sparsity. The model learns parameters a and b per phase:

scale_factor = a * exp(b * target_sparsity)

At inference time, the threshold is computed as:

threshold = scale_factor / sequence_length

Key benefits: - Target sparsity can be changed at runtime without recalibration - Threshold automatically adapts to sequence length - Supports independent prefill and decode phase calibration - Exponential model provides better fit (lower RMSE)

cache_dir: str | None

chunk_size: int

data_dir: str | None

fit_logspace: bool

max_seqlen: int

model_config = {'extra': 'forbid', 'validate_assignment': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

num_decode_tokens: int

num_length_bins: int

samples: int

target_sparse_ratio: dict[str, float]

threshold_trials: list[float] | None

classmethod validate_chunk_size(v): Validate chunk_size is positive or -1 (disabled).

classmethod validate_max_seqlen(v): Validate max_seqlen is at least 1024.

classmethod validate_num_decode_tokens(v): Validate num_decode_tokens is positive.

classmethod validate_num_length_bins(v): Validate num_length_bins is positive.

classmethod validate_samples(v): Validate samples is positive.

classmethod validate_target_sparse_ratio(v): Validate target sparsity ratio dict.

classmethod validate_threshold_trials(v): Validate threshold_trials are in valid range.

class FlashSkipSoftmaxConfig

Bases: SparseAttentionConfig

Configuration for Flash Attention-aware softmax skip sparse attention.

model_config = {'extra': 'forbid', 'validate_assignment': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

sparse_cfg: dict[str | Callable, dict[str, Any]]

class SparseAttentionAttributeConfig

Bases: ModeloptBaseConfig

Sparse attention attribute configuration for pattern-based module config.

backend: str

bc: int

br: int

collect_stats: bool

dense_recent_tokens: int

dense_sink_tokens: int

enable: bool

export_sparse_softmax: bool

initial_disabled_steps: int

is_causal: bool

method: str

model_config = {'extra': 'forbid', 'validate_assignment': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

skip_softmax_threshold: float

sparsity_m: int

sparsity_n: int

thresholds: dict[str, list[float]]

classmethod validate_backend(v): Validate backend is pytorch or triton.

classmethod validate_block_size(v): Validate block sizes are positive integers.

classmethod validate_dense_recent_tokens(v): Validate dense_recent_tokens is non-negative.

classmethod validate_dense_sink_tokens(v): Validate dense_sink_tokens is non-negative.

classmethod validate_method(v): Validate method is a string.

classmethod validate_sparsity_m(v): Validate sparsity_m is 4 or 8.

classmethod validate_sparsity_n(v): Validate sparsity_n is non-negative.

validate_sparsity_n_vs_m(): Validate sparsity_n is within the supported range for the given sparsity_m.

classmethod validate_thresholds(v): Validate thresholds is a dict of lists with valid phases and values in range (0, 1).

class SparseAttentionConfig

Bases: ModeloptBaseConfig

Base configuration for sparse attention optimization.

This base configuration provides the common structure for all sparse attention methods and supports pattern-based layer configuration.

export_format: str | None

model_config = {'extra': 'forbid', 'validate_assignment': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

sparse_cfg: dict[str | Callable, dict[str, Any]]

class VSAAttributeConfig

Bases: ModeloptBaseConfig

Video Sparse Attention (VSA) attribute configuration.

VSA uses a two-branch architecture optimized for video diffusion models: 1. Compression branch: Block-averaged coarse attention 2. Sparse branch: Top-K block selection for fine-grained attention

block_size_3d: tuple[int, int, int] | list[int]

collect_stats: bool

enable: bool

method: str

model_config = {'extra': 'forbid', 'validate_assignment': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

top_k_ratio: float

classmethod validate_block_size_3d(v): Validate 3D block size.

classmethod validate_top_k_ratio(v): Validate top-K ratio is in valid range.

classmethod validate_video_shape(v): Validate video shape if provided.

classmethod validate_vsa_method(v): Validate method is ‘vsa’.

video_shape: tuple[int, int, int] | list[int] | None

class VSAConfig

Bases: SparseAttentionConfig

Configuration for Video Sparse Attention optimization.

model_config = {'extra': 'forbid', 'validate_assignment': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

sparse_cfg: dict[str | Callable, dict[str, Any]]