RuntimeMemoryStatus

class tensorrt_llm.llmapi.RuntimeMemoryStatus(*, state: ~typing.Literal['running', 'parking', 'parked', 'waking', 'failed'], parked_tags: list[~tensorrt_llm.llmapi.llm_args.ExecutorMemoryType] = <factory>)

Bases: StrictBaseModel

Status of runtime memory managed by sleep and wakeup.

field parked_tags: list[ExecutorMemoryType] [Optional]

Memory tags that remain released, sorted lexicographically by value.

field state: Literal['running', 'parking', 'parked', 'waking', 'failed'] [Required]

Current runtime-memory admission state.

__init__(**data: Any) → None

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

validator sort_parked_tags  »  parked_tags