model_calib

Calibration utilities.

Functions

max_calibrate

Calibrate the model using max.

awq

Apply AWQ to the model.

smoothquant

Smooth-Quant variant with per-channel weight scaling.

svdquant

Lite version of SVDQuant.
awq(model, algorithm='awq_lite', forward_loop=None, **kwargs)

Apply AWQ to the model.

Parameters:

  • model (Module) –

  • algorithm (str) –

  • forward_loop (Callable[[Module], None] | None) –

max_calibrate(model, forward_loop=None, distributed_sync=True)

Calibrate the model using max.

Parameters:

  • model (Module) –

  • forward_loop (Callable[[Module], None] | None) –

smoothquant(model, forward_loop=None, alpha=1.0)

Smooth-Quant variant with per-channel weight scaling.

The parameters are as described in SmoothQuantCalibConfig.

Parameters:

  • model (Module) –

  • forward_loop (Callable[[Module], None] | None) –

svdquant(model, forward_loop=None, lowrank=32, **kwargs)

Lite version of SVDQuant.

The parameters are as described in SVDQuantConfig.

Parameters:

  • model (Module) – Model to be calibrated.

  • forward_loop (Callable[[Module], None] | None) – A callable which takes the model as argument and forwards calibration data through the model.

  • lowrank (int) –