convert_hf_config

Convert modelopt quantization export config to align with llm-compressor config format.

Functions

convert_hf_quant_config_format

Converts modelopt quantization config dictionary to align with llm-compressor config format.

convert_hf_quant_config_format(input_config)

Converts modelopt quantization config dictionary to align with llm-compressor config format.

Parameters:

input_config (dict) – The original quantization config dictionary.

Return type:

dict

Example

{
    "producer": {"name": "modelopt", "version": "0.19.0"},
    "quantization": {
        "quant_algo": "FP8",
        "kv_cache_quant_algo": "FP8",
        "exclude_modules": ["lm_head"],
    },
}

Returns:

A new dictionary in the target format.

Example (for FP8 input):

{
    "config_groups": {
        "group_0": {
            "input_activations": {"dynamic": False, "num_bits": 8, "type": "float"},
            "weights": {"dynamic": False, "num_bits": 8, "type": "float"},
        }
    },
    "ignore": ["lm_head"],
    "quant_algo": "FP8",
    "kv_cache_scheme": "FP8",
    "producer": {"name": "modelopt", "version": "0.29.0"},
}

Parameters:

input_config (dict)

Return type:

dict