hparam

Standard hyperparameter class for regular symbol.

Classes

Hparam

A base hyperparameter of a DynamicModule.

class Hparam

Bases: object

A base hyperparameter of a DynamicModule.

An example of such a Hparam could be an hparam with identity dependencies.

ActiveSlice

alias of Union[slice, LongTensor]

Importance

alias of Optional[Tensor]

ImportanceEstimator

alias of Callable[[], Optional[Tensor]]

__init__(choices, original=None)

Initializes Hparam with original value and choices.

Parameters:

• choices (Sequence[Tuple[int, ...] | int | float | CustomHPType]) –

• original (Tuple[int, ...] | int | float | CustomHPType | None) –

Return type:

None

property active: Tuple[int, ...] | int | float | CustomHPType

Return the currently active value.

property active_slice: slice | LongTensor

Return the currently active sorted indices or slice corresponding to the active value.

property choices: Sequence[Tuple[int, ...] | int | float | CustomHPType]

Return available choices.

enforce_order(order=None)

Store a reference to this order and enforce the order for active_slice.

This function enables the user to enforce an order how the active_slice is generated.

Example

If the hparam has active value 16 and the max value is 32, the active_slice by default will be slice(16), which is equivalent to range(16) (although faster). When order is set active_slice will instead return self._order[:16].

TODO: will we ever need a cycle detector here?

Parameters:

order (Tensor | None) –

Return type:

None

property importance: Tensor | None

Computes and returns the normalized importance among the features the hparam represents.

Note that the importance is represented as a 1d-tensor with the length equal to the max choice (Hparam.max) of the hparam.

For example, if the hparam represents the number of in_channels to a Conv2d layer, the importance should be a 1d-tensor of importance score with length equal to the number of in_channels.

Note that each module should register appropriate importance callbacks to compute the actual importance associated with the hparam choices. If there is no notion of importance for the hparam, this function returns None.

property is_configurable

Return whether the hparam is configurable.

property is_sortable

Return whether hparam in sortable.

property max: Tuple[int, ...] | int | float | CustomHPType

Return max value from among choices.

property min: Tuple[int, ...] | int | float | CustomHPType

Return min value from among choices.

property original: Tuple[int, ...] | int | float | CustomHPType

Return original value from among choices.

register_importance(importance_estimator)

Register importance estimator for the hparam.

This estimator does not take any arguments and should return a single argument ( optional 1d-tensor) representing the importance among features the hparam represents. If the return argument is a tensor, the length of the tensor must be equal to the max choice (Hparam.max) of the hparam.

Parameters:

importance_estimator (Callable[[], Tensor | None]) –