MambaStateConfig#

class tensorrt_llm.llmapi.MambaStateConfig(
*,
periodic_snapshot_interval: ~typing.Annotated[int,
~annotated_types.Ge(ge=0)] = 0,
additional_snapshot_offsets_from_start: ~typing.List[~typing.Annotated[int,
~pydantic.types.Strict(strict=True),
FieldInfo(annotation=NoneType,
required=True,
metadata=[Gt(gt=0)])]] = <factory>,
additional_snapshot_offsets_from_end: ~typing.List[~typing.Annotated[int,
~pydantic.types.Strict(strict=True),
FieldInfo(annotation=NoneType,
required=True,
metadata=[Ge(ge=0)])]] = <factory>,
enable_branch_snapshot: bool = False,
)[source]#

Bases: StrictBaseModel

Configuration for reusable Mamba recurrent-state snapshots.

field additional_snapshot_offsets_from_end: List[Annotated[int, Strict(strict=True), FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0)])]] [Optional]#

Additional Mamba state snapshot offsets measured backward from the end of each prompt. An offset of 0 selects the prompt end. Offsets that do not resolve inside the prompt are ignored. These snapshots require KV cache manager V2.

field additional_snapshot_offsets_from_start: List[Annotated[int, Strict(strict=True), FieldInfo(annotation=NoneType, required=True, metadata=[Gt(gt=0)])]] [Optional]#

Additional Mamba state snapshot offsets measured from the start of each prompt. Offsets beyond the prompt length are ignored. These snapshots require KV cache manager V2.

field enable_branch_snapshot: bool = False#

Snapshot the Mamba recurrent state where a request’s content diverges from the prefix cache, so that later requests sharing the same prefix can reuse up to the fork instead of being truncated to an earlier snapshot. If no periodic or additional snapshot point applies to the prompt, the prompt end is snapshotted as a fallback. Requires KV cache manager V2.

field periodic_snapshot_interval: Annotated[int, Ge(ge=0)] = 0#

The number of tokens between periodic snapshots in the Mamba prefix cache. Periodic snapshots are disabled by default; set this to a positive value to enable them.

Constraints:

  • ge = 0

__init__(**data: Any) → None#

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.