nvalchemi.training.distillation.hessian_distillation_fn#
- nvalchemi.training.distillation.hessian_distillation_fn(models, batch)[source]#
Run the student forward pass and add its Hessian-vector product.
The student’s Hessian-vector product is the product of its energy Hessian with the probe direction in
teacher_hvp_probe, the direction the teacher’s own product was labeled with. It is taken on a second pass narrowed to the student’s energy. The second pass is needed because a conservative model derives its forces from the autograd graph that the second derivative needs, and frees that graph outside training mode, so the stock forward’s energy cannot be differentiated again. Both derivatives are taken withcreate_graph=True, soHessianMatchingLosscan backpropagate through them. The batch’srequires_gradflags are restored, and the narrowed pass reuses the neighbor list the stock forward just ran on.- Parameters:
models (Mapping[str, BaseModelMixin]) – Named models of the strategy; only
"student"is read.batch (Batch) – Input batch, carrying the
teacher_hvp_probefield thehessianteacher signal writes.
- Returns:
The stock
predicted_*outputs pluspredicted_hvp.- Return type:
dict[str, torch.Tensor]
- Raises:
KeyError – If the batch carries no probe, which means it was never labeled with the
hessiansignal, or if the student computes no energy to differentiate.
Notes
On every frame the student trains on, the second pass adds the three backward passes
HessianOperatortakes, the last of them through a second-order graph held for the whole step. A stochastic student draws afresh in the narrowed pass, so its curvature is measured on a different realization than its energy.