CalibConfig#

class tensorrt_llm.llmapi.CalibConfig( *, device: Literal['cuda', 'cpu'] = 'cuda', calib_dataset: str = 'cnn_dailymail', calib_batches: int = 512, calib_batch_size: int = 1, calib_max_seq_length: int = 512, random_seed: int = 1234, tokenizer_max_seq_length: int = 2048, )[source]#

Bases: StrictBaseModel

Calibration configuration.

field calib_batch_size: int = 1#: The batch size that the calibration runs.

field calib_batches: int = 512#: The number of batches that the calibration runs.

field calib_dataset: str = 'cnn_dailymail'#: The name or local path of calibration dataset.

field calib_max_seq_length: int = 512#: The maximum sequence length that the calibration runs.

field device: Literal['cuda', 'cpu'] = 'cuda'#: The device to run calibration.

field random_seed: int = 1234#: The random seed used for calibration.

field tokenizer_max_seq_length: int = 2048#: The maximum sequence length to initialize tokenizer for calibration.

__init__(**data: Any) → None#

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.