fastnas

Module implementing fasnas pruning algorithm for search.

Classes

`BinarySearcher`	An iterative searcher that uses binary search to find the best configuration.
`FastNASPatchManager`	A patch manager for FastNAS (same as AutoNAS except no sampling during training).

Functions

`convert_fastnas_searchspace`	Convert search space for FastNAS mode with correct patch manager.
`restore_fastnas_searchspace`	Restore search space for FastNAS mode with correct patch manager.

class BinarySearcher

Bases: IterativeSearcher

An iterative searcher that uses binary search to find the best configuration.

after_step()

Update boundaries of the interval after recursing.

Return type:: None

before_search()

Build sensitivity map before search that we use to approximate the cost function.

Return type:: None

before_step()

Check what the middle value is to determine where we recurse.

Return type:: None

property default_state_dict: Dict[str, Any]: We also store the sensitivity map and related arguments.

early_stop()

Early stop if the interval is small enough.

Return type:: bool

property hparam_names_for_search: Set[str]: We can only optimize over certain types of hparams in binary search.

property hparam_types_for_search: Tuple[Type]: We can only optimize over certain types of hparams in binary search.

load_search_checkpoint()

We only want to load sensitivity map and original_score here and keep the rest.

Return type:: bool

max_degrade: float

middle_value: float

min_degrade: float

original_score: float

sample()

Check in which interval we should recurse and sets the corresponding subnet.

Return type:: Dict[str, Any]

sensitivity_map: Dict[str, Dict[int, float]]

class FastNASPatchManager

Bases: AutoNASPatchManager

A patch manager for FastNAS (same as AutoNAS except no sampling during training).

property sample_during_training: Indicates whether we should sample a new subnet during training.

convert_fastnas_searchspace(model, config)

Convert search space for FastNAS mode with correct patch manager.

Parameters:

model (Module) –
config (ModeloptBaseConfig) –

Return type:

Tuple[Module, Dict[str, Any]]

restore_fastnas_searchspace(model, config, metadata)

Restore search space for FastNAS mode with correct patch manager.

Parameters:

model (Module) –
config (ModeloptBaseConfig) –
metadata (Dict[str, Any]) –

Return type:

Module