kv_cache_auto_quant#
Layer-wise KV-cache AutoQuantize using isolated forward KL sensitivity.
Classes
KV-cache AutoQuantize backend using the shared search/checkpoint lifecycle. |
- class AutoQuantizeKVSearcher#
Bases:
BaseSearcherKV-cache AutoQuantize backend using the shared search/checkpoint lifecycle.
- before_search()#
Resolve attention decisions and calibrate or restore candidate scales.
- Return type:
None
- property default_search_config: dict[str, Any]#
Return KV-specific fields layered on the shared search configuration.
- property default_state_dict: dict[str, Any]#
Return the checkpointed KV search state.
- load_search_checkpoint()#
Load compatible fields before validating the resolved KV search signature.
- Return type:
bool
- method_name = 'kl_div'#
- run_search()#
Score candidates when needed, solve the budget, and apply the selection.
- Return type:
None
- sanitize_search_config(config)#
Validate the data inputs required by isolated forward-KL scoring.
- Parameters:
config (dict[str, Any] | None)
- Return type:
dict[str, Any]