nvalchemi.training.hooks.TrainableParameterHook

pydantic model nvalchemi.training.hooks.TrainableParameterHook

Select which parameters stay trainable during fine-tuning.

On registration with a TrainingStrategy, freeze_patterns and trainable_patterns (globs over fully-qualified parameter names) resolve to the trainable set. What you set decides the behaviour:

• trainable_patterns only — train exactly those; freeze the rest.

• freeze_patterns only — freeze those; train the rest.

• both — freeze the freeze_patterns set, but trainable_patterns win: a parameter they match stays trainable even if a freeze pattern also matches it.

Frozen parameters are temporarily marked requires_grad=False during run and restored afterward. Set freeze_mode="optimizer_only" to instead keep them out of the optimizer while preserving their gradients.

Raises:

ValueError – If no patterns are supplied, or if any pattern matches no parameter.

Warns:

UserWarning – If registered after optimizers already exist. The stored filter is updated, but existing optimizer parameter groups are not rebuilt.

Examples

>>> from nvalchemi.training.hooks import TrainableParameterHook
>>> TrainableParameterHook(
...     freeze_patterns=("main.model.*",),
...     trainable_patterns=("main.model.projection.*",),
... ).frequency
1

field freeze_patterns: tuple[str, ...] = ()

Glob patterns for parameters to freeze. Overridden by trainable_patterns — a parameter matching both stays trainable.

field trainable_patterns: tuple[str, ...] = ()

Glob patterns for parameters to keep trainable. On their own they define the full trainable set (everything else is frozen).

field freeze_mode: Literal['requires_grad', 'optimizer_only'] = 'requires_grad'

Whether excluded parameters are temporarily frozen via requires_grad=False or only excluded from optimizer construction.