nvalchemi.training.distillation.HessianMatchingLoss#
- class nvalchemi.training.distillation.HessianMatchingLoss(*, target_key='teacher_hvp', prediction_key='predicted_hvp', normalize_by_atom_count=True, ignore_nonfinite=True, dtype_policy='strict')[source]#
Mean-squared-error loss on Hessian-vector products.
Energies and forces pin down the value and slope of the student’s potential-energy surface. Its curvature decides vibrational spectra, the stiffness of a minimum, and whether an integrator stays stable at a given timestep. This term supervises that curvature without forming a Hessian. It compares the teacher’s and student’s Hessian-vector products, the product of each model’s energy Hessian with one random probe direction \(\mathbf{v}\). The per-component residual is
\[\rho_{ia\alpha} = \left( (\hat{\mathbf{H}}\mathbf{v})_{ia\alpha} - (\mathbf{H}\mathbf{v})_{ia\alpha} \right)^2,\]and each product costs a few backward passes per model rather than \(3V\). The residuals are reduced the way force residuals are, according to
normalize_by_atom_count. Thehessianteacher signal writes the teacher’s product toteacher_hvpand the probe toteacher_hvp_probe.hessian_distillation_fn()computes the student’s product along that same probe.- Parameters:
target_key (str, default "teacher_hvp") – Target container key for the teacher’s Hessian-vector product.
prediction_key (str, default "predicted_hvp") – Prediction container key for the student’s Hessian-vector product.
normalize_by_atom_count (bool, default True) – When
True, compute a mean squared residual per graph, then mean over graphs. WhenFalse, compute one global mean over valid components.ignore_nonfinite (bool, default True) – When
True, components whose target isNaNor infinite are excluded from both loss value and gradient. It defaults on because a marginally stable model overflows first in its second derivatives.dtype_policy ({"strict", "prediction_to_target", "target_to_prediction"}, default "strict") – How to handle prediction/target dtype mismatches before validation.
- Raises:
ValueError – If the graph-balanced reduction is requested without
batch_idxandnum_graphsmetadata.
Examples
>>> import torch >>> from nvalchemi.training.distillation import HessianMatchingLoss >>> loss_fn = HessianMatchingLoss() >>> pred = torch.tensor([[2.0, 0.0, 0.0], [0.0, 0.0, 0.0]]) >>> target = torch.zeros(2, 3) >>> batch_idx = torch.tensor([0, 1]) >>> loss_fn(pred, target, batch_idx=batch_idx, num_graphs=2) tensor(0.6667)
See also
nvalchemi.models.HessianOperatorThe shared estimator.
Notes
requires_eval_gradisTruebecause the prediction is a second derivative. Validation therefore runs with gradients enabled and costs the same two student passes as a training step. One probe constrains one direction, so coverage of the curvature comes from redrawing the probe. An on-policy run draws a fresh probe every time it labels a frame, whereas a store labeled once freezes one direction per structure.Each probe component is standard normal, which makes the graph-balanced value a Hutchinson estimate of \(\frac{1}{B}\sum_g \lVert \Delta\mathbf{H}_g \rVert_F^2 / 3V_g\) in (eV/A^2)^2. For a near-converged student, this value is one to two orders of magnitude above the force mean-squared error. It is also a one-sample estimate whose relative spread is of order one. Start the term a hundred to ten thousand times lighter than the force term, and treat a single batch’s value as noise.
The term matches the curvature of the energy. A direct-force model, teacher or student, has a well-defined energy Hessian, but that Hessian is not the derivative of the forces distilled beside it. For a direct-force student, the term therefore supervises the energy head alone.
- compute_residual(pred, target, valid)[source]#
Return squared component residuals, zeroing invalid components.
- Parameters:
pred (Float[Tensor, 'V 3'])
target (Float[Tensor, 'V 3'])
valid (Bool[Tensor, 'V 3'])
- Return type:
Float[Tensor, ‘V 3’]
- mask(pred, target, ctx, **kwargs)[source]#
Return one validity flag per Cartesian component.
- Parameters:
pred (Float[Tensor, 'V 3'])
target (Float[Tensor, 'V 3'])
ctx (ReductionContext)
kwargs (Any)
- Return type:
Bool[Tensor, ‘V 3’]
- reduce(residual, valid, ctx, **kwargs)[source]#
Reduce squared component residuals to a scalar loss.
masked_mean()takes the global mean over valid entries, or the graph-balanced one whennormalize_by_atom_countis set, in at least float32 either way.- Parameters:
residual (Float[Tensor, 'V 3'])
valid (Bool[Tensor, 'V 3'])
ctx (ReductionContext)
kwargs (Any)
- Return type: