model_calib

Calibration utilities.

Functions

max_calibrate

Calibrate the model using max.

awq

Apply AWQ to the model.

smoothquant

Smooth-Quant variant with per-channel weight scaling.

svdquant

Lite version of SVDQuant.
awq(model, forward_loop=None, algorithm='awq_lite', **kwargs)

Apply AWQ to the model.

Parameters:

  • model (Module) – Model to be calibrated.

  • forward_loop (Callable[[Module], None] | None) – A callable which takes the model as argument and forwards calibration data through the model.

  • algorithm (str) –

See AWQFullCalibConfig for details on the remaining arguments.

max_calibrate(model, forward_loop=None, distributed_sync=True)

Calibrate the model using max.

Parameters:

  • model (Module) – Model to be calibrated.

  • forward_loop (Callable[[Module], None] | None) – A callable which takes the model as argument and forwards calibration data through the model.

See MaxCalibConfig for details on the remaining arguments.

smoothquant(model, forward_loop=None, alpha=1.0)

Smooth-Quant variant with per-channel weight scaling.

Parameters:

  • model (Module) – Model to be calibrated.

  • forward_loop (Callable[[Module], None] | None) – A callable which takes the model as argument and forwards calibration data through the model.

See SmoothQuantCalibConfig for details on the remaining arguments.

svdquant(model, forward_loop=None, lowrank=32, **kwargs)

Lite version of SVDQuant.

Parameters:

  • model (Module) – Model to be calibrated.

  • forward_loop (Callable[[Module], None] | None) – A callable which takes the model as argument and forwards calibration data through the model.

  • lowrank (int) –

See SVDQuantConfig for details on the remaining arguments.