mode

This module contains the mode descriptor for the quantization mode.

Classes

`EagleModeDescriptor`	Class to describe the `"eagle"` mode.
`MTPModeDescriptor`	Class to describe the `"mtp"` mode.
`MedusaModeDescriptor`	Class to describe the `"medusa"` mode.

class EagleModeDescriptor

Bases: ModeDescriptor

Class to describe the "eagle" mode.

The properties of this mode can be inspected via the source code.

property config_class: type[ModeloptBaseConfig]: Specifies the config class for the mode.

property convert: Callable[[Module, ModeloptBaseConfig], tuple[Module, dict[str, Any]]] | Callable[[Module, ModeloptBaseConfig, Any], tuple[Module, dict[str, Any]]]: The mode’s entrypoint for converting a model.

property restore: Callable[[Module, ModeloptBaseConfig, dict[str, Any]], Module]: The mode’s entrypoint for restoring a model.

class MTPModeDescriptor

Bases: ModeDescriptor

Class to describe the "mtp" mode.

The properties of this mode can be inspected via the source code.

property config_class: type[ModeloptBaseConfig]: Specifies the config class for the mode.

property convert: Callable[[Module, ModeloptBaseConfig], tuple[Module, dict[str, Any]]] | Callable[[Module, ModeloptBaseConfig, Any], tuple[Module, dict[str, Any]]]: The mode’s entrypoint for converting a model.

property restore: Callable[[Module, ModeloptBaseConfig, dict[str, Any]], Module]: The mode’s entrypoint for restoring a model.

class MedusaModeDescriptor

Bases: ModeDescriptor

Class to describe the "medusa" mode.

The properties of this mode can be inspected via the source code.

property config_class: type[ModeloptBaseConfig]: Specifies the config class for the mode.

property convert: Callable[[Module, ModeloptBaseConfig], tuple[Module, dict[str, Any]]] | Callable[[Module, ModeloptBaseConfig, Any], tuple[Module, dict[str, Any]]]: The mode’s entrypoint for converting a model.

property restore: Callable[[Module, ModeloptBaseConfig, dict[str, Any]], Module]: The mode’s entrypoint for restoring a model.