nvalchemi.training.distillation.hessian_distillation_fn#

nvalchemi.training.distillation.hessian_distillation_fn(models, batch)[source]#

Run the student forward pass and add its Hessian-vector product.

The student’s Hessian-vector product is the product of its energy Hessian with the probe direction in teacher_hvp_probe, the direction the teacher’s own product was labeled with. It is taken on a second pass narrowed to the student’s energy. The second pass is needed because a conservative model derives its forces from the autograd graph that the second derivative needs, and frees that graph outside training mode, so the stock forward’s energy cannot be differentiated again. Both derivatives are taken with create_graph=True, so HessianMatchingLoss can backpropagate through them. The batch’s requires_grad flags are restored, and the narrowed pass reuses the neighbor list the stock forward just ran on.

Parameters:
  • models (Mapping[str, BaseModelMixin]) – Named models of the strategy; only "student" is read.

  • batch (Batch) – Input batch, carrying the teacher_hvp_probe field the hessian teacher signal writes.

Returns:

The stock predicted_* outputs plus predicted_hvp.

Return type:

dict[str, torch.Tensor]

Raises:

KeyError – If the batch carries no probe, which means it was never labeled with the hessian signal, or if the student computes no energy to differentiate.

Notes

On every frame the student trains on, the second pass adds the three backward passes HessianOperator takes, the last of them through a second-order graph held for the whole step. A stochastic student draws afresh in the narrowed pass, so its curvature is measured on a different realization than its energy.