nvalchemi.training.peft.lora_hook.LoRAHook#
- class nvalchemi.training.peft.lora_hook.LoRAHook(*, lora_config, register_parameters=True)[source]#
Apply LoRA adapters to models.
This hook is automatically prepended when
FineTuningStrategyreceives aLoRAConfig.- Parameters:
lora_config (LoRAConfig) – LoRA PEFT configuration describing adapter targets, rank, scaling, dropout, MLP wrapping, and custom wrapper registrations.
register_parameters (bool, optional) – If
True, register LoRA adapter parameters as trainable and managed on the workflow. Set toFalsefor standalone model loading where no training workflow registry exists. Defaults toTrue.
- frequency#
Required by the hook protocol; always
1.- Type:
int
- stage#
This hook does not run at training stages.
- Type:
None
- model_config = {'arbitrary_types_allowed': True}#
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- on_register(workflow)[source]#
Register wrappers, inject adapters, and register adapter parameters.
The public configuration is validated before creating this hook. During registration, model-prefixed targets are converted to model-local targets for each model to be compatible with the PhysicsNeMo
apply_lorafunction. For example,"student.model.projection"becomes the PhysicsNeMotarget_modulesentry"model.projection"for the"student"model. Adapters are then injected, and their trainable and managed parameter names are registered on the workflow.- Parameters:
workflow (Any)
- Return type:
None