autonas

Entrypoints for AutoNAS mode.

Classes

`AutoNASConfig`	Configuration for the `"autonas"` mode.
`AutoNASModeDescriptor`	Class to describe the `"autonas"` mode.
`AutoNASPatchManager`	A class to handle the monkey patching of the model for automode.
`EvolveSearcher`	An iterative searcher that uses an evolutionary algorithm to optimize the subnet config.
`IterativeSearcher`	Base class for iterative search algorithms.
`RandomSearcher`	An iterative searcher that samples subnets randomly.

Functions

`convert_autonas_searchspace`	Convert search space for AutoNAS mode with correct patch manager.
`convert_searchspace`	Convert given model into a search space.
`restore_autonas_searchspace`	Restore search space for AutoNAS mode with correct patch manager.
`restore_searchspace`	Restore a search space from the given model.
`update_autonas_metadata`	Update subnet config to current subnet config of model.

class AutoNASConfig

Bases: ModeloptBaseRuleConfig

Configuration for the "autonas" mode.

model_config = {'extra': 'allow', 'validate_assignment': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

nn_batchnorm1d: DynamicBatchNorm1dConfig | None | dict[str, DynamicBatchNorm1dConfig | None]

nn_batchnorm2d: DynamicBatchNorm2dConfig | None | dict[str, DynamicBatchNorm2dConfig | None]

nn_batchnorm3d: DynamicBatchNorm3dConfig | None | dict[str, DynamicBatchNorm3dConfig | None]

nn_conv1d: DynamicConv1dConfig | None | dict[str, DynamicConv1dConfig | None]

nn_conv2d: DynamicConv2dConfig | None | dict[str, DynamicConv2dConfig | None]

nn_conv3d: DynamicConv3dConfig | None | dict[str, DynamicConv3dConfig | None]

nn_convtranspose1d: DynamicConvTranspose1dConfig | None | dict[str, DynamicConvTranspose1dConfig | None]

nn_convtranspose2d: DynamicConvTranspose2dConfig | None | dict[str, DynamicConvTranspose2dConfig | None]

nn_convtranspose3d: DynamicConvTranspose3dConfig | None | dict[str, DynamicConvTranspose3dConfig | None]

nn_groupnorm: DynamicGroupNormConfig | None | dict[str, DynamicGroupNormConfig | None]

nn_instancenorm1d: DynamicInstanceNorm1dConfig | None | dict[str, DynamicInstanceNorm1dConfig | None]

nn_instancenorm2d: DynamicInstanceNorm2dConfig | None | dict[str, DynamicInstanceNorm2dConfig | None]

nn_instancenorm3d: DynamicInstanceNorm3dConfig | None | dict[str, DynamicInstanceNorm3dConfig | None]

nn_layernorm: DynamicLayerNormConfig | None | dict[str, DynamicLayerNormConfig | None]

nn_linear: DynamicLinearConfig | None | dict[str, DynamicLinearConfig | None]

nn_sequential: DynamicSequentialConfig | None | dict[str, DynamicSequentialConfig | None]

nn_syncbatchnorm: DynamicSyncBatchNormConfig | None | dict[str, DynamicSyncBatchNormConfig | None]

class AutoNASModeDescriptor

Bases: ModeDescriptor

Class to describe the "autonas" mode.

The properties of this mode can be inspected via the source code.

property config_class: type[ModeloptBaseConfig]: Specifies the config class for the mode.

property convert: Callable[[Module, ModeloptBaseConfig], tuple[Module, dict[str, Any]]] | Callable[[Module, ModeloptBaseConfig, Any], tuple[Module, dict[str, Any]]]: The mode’s entrypoint for converting a model.

property export_mode: str | None: The mode that corresponds to the export mode of this mode.

property name: str: Returns the value (str representation) of the mode.

property next_modes: set[str] | None: Modes that must immediately follow this mode.

property restore: Callable[[Module, ModeloptBaseConfig, dict[str, Any]], Module]: The mode’s entrypoint for restoring a model.

property search_algorithm: type[BaseSearcher]: Specifies the search algorithm to use for this mode (if any).

property update_for_new_mode: Callable[[Module, ModeloptBaseConfig, dict[str, Any]], None]: The mode’s entrypoint for updating the models state before new mode.

property update_for_save: Callable[[Module, ModeloptBaseConfig, dict[str, Any]], None]: The mode’s entrypoint for updating the models state before saving.

class AutoNASPatchManager

Bases: PatchManager

A class to handle the monkey patching of the model for automode.

property sample_during_training: bool: Indicates whether we should sample a new subnet during training.

class EvolveSearcher

Bases: IterativeSearcher

An iterative searcher that uses an evolutionary algorithm to optimize the subnet config.

after_step()

Update population after each iterative step.

Return type:: None

before_search()

Set the lower bound of the constraints to 0.85 * upper bound before search.

Return type:: None

before_step()

Update candidates and population before each iterative step.

Return type:: None

candidates: list[dict[str, Any]]

property default_search_config: dict[str, Any]: Default search config contains additional algorithm parameters.

property default_state_dict: dict[str, Any]: Return default state dict.

population: list[dict[str, Any]]

sample()

Sampling a new subnet involves random sampling, mutation, and crossover.

Return type:: dict[str, Any]

class IterativeSearcher

Bases: BaseSearcher, ABC

Base class for iterative search algorithms.

after_search()

Select best model.

Return type:: None

after_step()

Run after each iterative step.

Return type:: None

before_search()

Ensure that the model is actually configurable and ready for eval.

Return type:: None

before_step()

Run before each iterative step.

Return type:: None

best: dict[str, Any]

best_history: dict[str, Any]

candidate: dict[str, Any]

constraints_func: ConstraintsFunc

property default_search_config: dict[str, Any]: Get the default config for the searcher.

property default_state_dict: dict[str, Any]: Return default state dict.

early_stop()

Check if we should early stop the search if possible.

Return type:: bool

history: dict[str, Any]

iter_num: int

num_satisfied: int

run_search()

Run iterative search loop.

Return type:: None

run_step()

The main routine of each iterative step.

Return type:: None

abstractmethod sample()

Sample and select new sub-net configuration and return configuration.

Return type:: dict[str, Any]

samples: dict[str, Any]

sanitize_search_config(config)

Sanitize the search config dict.

Parameters:: config (dict[str, Any] | None)
Return type:: dict[str, Any]

class RandomSearcher

Bases: IterativeSearcher

An iterative searcher that samples subnets randomly.

sample()

Random sample new subset during each steo.

Return type:: dict[str, Any]

convert_autonas_searchspace(model, config)

Convert search space for AutoNAS mode with correct patch manager.

Parameters:

model (Module)
config (ModeloptBaseConfig)

Return type:

tuple[Module, dict[str, Any]]

convert_searchspace(model, config, patch_manager_type)

Convert given model into a search space.

Parameters:

model (Module)
config (ModeloptBaseConfig)
patch_manager_type (type[PatchManager])

Return type:

tuple[Module, dict[str, Any]]

restore_autonas_searchspace(model, config, metadata)

Restore search space for AutoNAS mode with correct patch manager.

Parameters:

model (Module)
config (ModeloptBaseConfig)
metadata (dict[str, Any])

Return type:

Module

restore_searchspace(model, config, metadata, patch_manager)

Restore a search space from the given model.

Parameters:

model (Module)
config (ModeloptBaseConfig)
metadata (dict[str, Any])
patch_manager (type[PatchManager])

Return type:

Module

update_autonas_metadata(model, config, metadata)

Update subnet config to current subnet config of model.

Parameters:

model (Module)
config (ModeloptBaseConfig)
metadata (dict[str, Any])

Return type:

None