SleepConfig

class tensorrt_llm.llmapi.SleepConfig(

*,

restore_modes: dict[~tensorrt_llm.llmapi.llm_args.ExecutorMemoryType,

~typing.Literal['NONE',

'MEMSET',

'CPU',

'PINNED'] | ~enum.Enum] = <factory>,

)

Bases: StrictBaseModel

Configuration for the LLM sleep/wakeup feature.

field restore_modes: dict[ExecutorMemoryType, Literal['NONE', 'MEMSET', 'CPU', 'PINNED'] | Enum] [Optional]

Per-component RestoreMode for the sleep feature. Keys are ExecutorMemoryType values (e.g. ‘model’, ‘kv_cache’), values can be RestoreMode names (NONE, MEMSET, CPU, PINNED) or RestoreMode enum values. Unlisted entries default to the suitable mode selected between PINNED and CPU.

__init__(**data: Any) → None

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

DEFAULT_RESTORE_MODES: ClassVar[dict[str, str]] = {ExecutorMemoryType.KV_CACHE: 'NONE'}