nvalchemi.training.distillation.HessianMatchingLoss#

class nvalchemi.training.distillation.HessianMatchingLoss(*, target_key='teacher_hvp', prediction_key='predicted_hvp', normalize_by_atom_count=True, ignore_nonfinite=True, dtype_policy='strict')[source]#

Mean-squared-error loss on Hessian-vector products.

Energies and forces pin down the value and slope of the student’s potential-energy surface. Its curvature decides vibrational spectra, the stiffness of a minimum, and whether an integrator stays stable at a given timestep. This term supervises that curvature without forming a Hessian. It compares the teacher’s and student’s Hessian-vector products, the product of each model’s energy Hessian with one random probe direction \(\mathbf{v}\). The per-component residual is

\[\rho_{ia\alpha} = \left( (\hat{\mathbf{H}}\mathbf{v})_{ia\alpha} - (\mathbf{H}\mathbf{v})_{ia\alpha} \right)^2,\]

and each product costs a few backward passes per model rather than \(3V\). The residuals are reduced the way force residuals are, according to normalize_by_atom_count. The hessian teacher signal writes the teacher’s product to teacher_hvp and the probe to teacher_hvp_probe. hessian_distillation_fn() computes the student’s product along that same probe.

Parameters:
  • target_key (str, default "teacher_hvp") – Target container key for the teacher’s Hessian-vector product.

  • prediction_key (str, default "predicted_hvp") – Prediction container key for the student’s Hessian-vector product.

  • normalize_by_atom_count (bool, default True) – When True, compute a mean squared residual per graph, then mean over graphs. When False, compute one global mean over valid components.

  • ignore_nonfinite (bool, default True) – When True, components whose target is NaN or infinite are excluded from both loss value and gradient. It defaults on because a marginally stable model overflows first in its second derivatives.

  • dtype_policy ({"strict", "prediction_to_target", "target_to_prediction"}, default "strict") – How to handle prediction/target dtype mismatches before validation.

Raises:

ValueError – If the graph-balanced reduction is requested without batch_idx and num_graphs metadata.

Examples

>>> import torch
>>> from nvalchemi.training.distillation import HessianMatchingLoss
>>> loss_fn = HessianMatchingLoss()
>>> pred = torch.tensor([[2.0, 0.0, 0.0], [0.0, 0.0, 0.0]])
>>> target = torch.zeros(2, 3)
>>> batch_idx = torch.tensor([0, 1])
>>> loss_fn(pred, target, batch_idx=batch_idx, num_graphs=2)
tensor(0.6667)

See also

nvalchemi.models.HessianOperator

The shared estimator.

Notes

requires_eval_grad is True because the prediction is a second derivative. Validation therefore runs with gradients enabled and costs the same two student passes as a training step. One probe constrains one direction, so coverage of the curvature comes from redrawing the probe. An on-policy run draws a fresh probe every time it labels a frame, whereas a store labeled once freezes one direction per structure.

Each probe component is standard normal, which makes the graph-balanced value a Hutchinson estimate of \(\frac{1}{B}\sum_g \lVert \Delta\mathbf{H}_g \rVert_F^2 / 3V_g\) in (eV/A^2)^2. For a near-converged student, this value is one to two orders of magnitude above the force mean-squared error. It is also a one-sample estimate whose relative spread is of order one. Start the term a hundred to ten thousand times lighter than the force term, and treat a single batch’s value as noise.

The term matches the curvature of the energy. A direct-force model, teacher or student, has a well-defined energy Hessian, but that Hessian is not the derivative of the forces distilled beside it. For a direct-force student, the term therefore supervises the energy head alone.

compute_residual(pred, target, valid)[source]#

Return squared component residuals, zeroing invalid components.

Parameters:
  • pred (Float[Tensor, 'V 3'])

  • target (Float[Tensor, 'V 3'])

  • valid (Bool[Tensor, 'V 3'])

Return type:

Float[Tensor, ‘V 3’]

extra_repr()[source]#

Human-readable hyperparameter summary for nn.Module’s repr.

Return type:

str

mask(pred, target, ctx, **kwargs)[source]#

Return one validity flag per Cartesian component.

Parameters:
  • pred (Float[Tensor, 'V 3'])

  • target (Float[Tensor, 'V 3'])

  • ctx (ReductionContext)

  • kwargs (Any)

Return type:

Bool[Tensor, ‘V 3’]

reduce(residual, valid, ctx, **kwargs)[source]#

Reduce squared component residuals to a scalar loss.

masked_mean() takes the global mean over valid entries, or the graph-balanced one when normalize_by_atom_count is set, in at least float32 either way.

Parameters:
  • residual (Float[Tensor, 'V 3'])

  • valid (Bool[Tensor, 'V 3'])

  • ctx (ReductionContext)

  • kwargs (Any)

Return type:

Tensor