ColdPageQuantizationCompressionConfig#

class tensorrt_llm.llmapi.ColdPageQuantizationCompressionConfig(
*,
algorithm: Literal['quantization_for_cold_page'] = 'quantization_for_cold_page',
quant: Literal['nvfp4'] = 'nvfp4',
skip_rope_quantization: bool = False,
nvfp4_residual_dim: Literal[0, 64] = 64,
scale_checkpoint_path: Annotated[str | None, MinLen(min_length=1)] = None,
)[source]#

Bases: KvCacheCompressionConfig

Quantize Host and Disk KV pages without changing the active GPU cache.

field algorithm: Literal['quantization_for_cold_page'] = 'quantization_for_cold_page'#
field nvfp4_residual_dim: Literal[0, 64] = 64#

64 (default): use two independently scaled FP4 components for the 64 DeepSeek-V4 target cold-page RoPE values: a main component and its residual. 0: use a single NVFP4 component. Only 0 and 64 are supported. Ignored for other models, draft caches, and when skip_rope_quantization preserves the original RoPE precision.

field quant: Literal['nvfp4'] = 'nvfp4'#

Quantization format stored in the compressed cache tier.

field scale_checkpoint_path: str | None = None#

Optional local ModelOpt NVFP4 checkpoint directory supplying per-layer K/V global scales. Omit it to use identity global scales.

Constraints:

  • min_length = 1

field skip_rope_quantization: bool = False#

True: preserve the position-encoded (RoPE) part of each K vector in its original active-cache precision and quantize the rest to NVFP4. False (default): quantize both parts, using residual RoPE quantization for DeepSeek-V4 when nvfp4_residual_dim is 64. An option to explore; measure its accuracy effect on your model.

__init__(
**data: Any,
) → None#

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

supports_block_reuse() → bool[source]#

Block reuse is unchanged because token identity is preserved.

supports_speculative_decoding() → bool[source]#

Target and draft KVCMs encode their own cold pages independently.

changes_physical_kv_length: ClassVar[bool] = False#

Whether physical and logical KV lengths can diverge.