nvalchemi.training.distillation.InProcessTeacherScorer#

class nvalchemi.training.distillation.InProcessTeacherScorer(teacher, signals, *, dtype=None, neighbor_list='rebuild', autocast=False, probe_seed=None)[source]#

Score a batch with a teacher model loaded in the current process.

The scorer owns the teacher’s evaluation contract: it narrows active_outputs to the outputs the requested signals need, builds and afterwards restores whatever neighbor list the teacher requires — or, on request, consumes the one the batch already carries — picks the grad mode the teacher’s autograd outputs need, runs the pass under its autocast setting, detaches every result, and normalizes each signal to its canonical shape. The batch is left exactly as it was found, so a scorer can be called mid-training on a live batch.

Each signal is a TeacherSignal mapping one teacher output to one batch field at one level. The built-in ones (BUILTIN_SIGNALS) are requested by name: energy to teacher_energy (B, 1) and stress to teacher_stress (B, 3, 3) at system level; forces to teacher_forces (V, 3), atomic_energies to teacher_atomic_energies (V,), embeddings (from compute_embeddings()) to teacher_node_embeddings (V, D), and hessian (from label_hvp()) to teacher_hvp (V, 3) at node level. The hessian signal is the teacher’s Hessian-vector product, the product of its energy Hessian with a random probe direction; that direction is written to teacher_hvp_probe. Any other teacher output is requested as a TeacherSignal of its own. The resolved specs are published as signal_specs and the fields they write as label_fields.

Parameters:
  • teacher (BaseModelMixin) – Model wrapper producing the signals. Placed in evaluation mode at construction and for the duration of every label() call, with the mode it arrived in restored afterwards; its parameters and their requires_grad flags are never modified.

  • signals (Iterable[str | TeacherSignal]) – Signals to produce: built-in names or TeacherSignal specs, each naming an output in the teacher’s declared outputs.

  • dtype (torch.dtype | None, optional) – Cast floating-point outputs to this dtype. Any floating-point dtype is accepted; whether a store can hold it is checked by label_dataset(), and a labeled store reads back at the reading dataset’s positions dtype regardless. Default None (keep the teacher’s dtype).

  • neighbor_list (NeighborListPolicy, optional) – "rebuild" builds the teacher’s own neighbor list for every call, hiding whatever list the batch carries, and rolls it back afterwards. "reuse" hands the teacher the batch’s list instead: the batch must carry the keys the teacher’s format reads, and a cutoff stamp, when the batch has one, must equal the teacher’s; otherwise label() raises rather than falling back. Whether the list holds each pair once or twice is not recorded on the batch, so a reused list must match the teacher’s half_list by construction. Default "rebuild".

  • autocast (bool | torch.dtype | None, optional) – Autocast mode for the scoring pass. The ambient autocast region is whatever autocast state is in force where label() is called, such as a caller’s AMP region. False disables autocast for the pass, so an ambient region never reaches the teacher. None leaves the ambient state untouched, so the teacher runs under the caller’s region when one is open. True or a floating-point torch.dtype enables autocast for the pass, whether or not an ambient region is open: a dtype sets the autocast dtype, and True keeps the autocast dtype in force for the device, which is the device default when no region is open and the ambient region’s dtype when one is. dtype applies after this setting: the teacher produces each label at the precision this mode gives, and dtype, when set, then casts it. Default False.

  • probe_seed (int | None, optional) – Seed of the generator the hessian probe direction is drawn from. Default None draws from the global RNG, a fresh direction per labeling. A call to label() can override it for that labeling alone; see the Notes.

Raises:

ValueError – If signals is empty, names a signal that is neither built in nor a TeacherSignal, gives two specs one name or one field, names a model output the teacher does not declare, gives a custom spec no model output, requests "embeddings" from a teacher that publishes no node-embedding shape, requests "hessian" from a teacher that declares no energy output, dtype is not a floating-point dtype, neighbor_list is neither "rebuild" nor "reuse", autocast is neither None, a bool, nor a floating-point dtype, or teacher is a composition planning more than one neighbor-list source.

Examples

>>> from nvalchemi.training.distillation import InProcessTeacherScorer
>>> scorer = InProcessTeacherScorer(teacher, ["energy", "forces"])
>>> labels = scorer.label(batch)
>>> labels["teacher_forces"][1]
'node'

A teacher output the built-in table does not cover is labeled through a spec of its own:

>>> from nvalchemi.training.distillation import TeacherSignal
>>> charges = TeacherSignal("charges", "charges", "teacher_charges", "node")
>>> scorer = InProcessTeacherScorer(teacher, ["energy", charges])
>>> scorer.label_fields
('teacher_charges', 'teacher_energy')

Notes

Under the default neighbor_list="rebuild" the teacher’s list is built for the forward pass and rolled back, and a list a composed pipeline keeps as an instance attribute, along with its captured per-source table, is hidden from the teacher for the whole of scoring. "reuse" is for the case where the student has already built the list the teacher needs, in the teacher’s format and at its cutoff, and one build per step is one too many; the scorer then checks only what it cannot infer and refuses the batch by name when the list is missing or stamped with another cutoff. A teacher composition planning more than one neighbor-list source is refused at construction, because the scorer builds one list per batch; compose it to plan a single list instead (neighbor_adaptation="always" or a large enough max_cutoff_ratio). requires_grad on positions and the teacher’s autograd inputs is restored after each call, as is every field a composed teacher writes onto the batch to wire one stage into the next.

Label precision is the scorer’s decision. Every consumer in the package calls label() inside the ambient autocast region and opens none of its own, so the autocast setting decides which autocast mode the teacher runs under. The default disables autocast, so a label taken during a mixed-precision training or generation phase equals the one label_dataset() writes offline. Inside a caller’s region, True and a dtype differ: True enables autocast at the region’s dtype, whereas a dtype pins the pass to itself.

Forward-pass signals share one teacher pass. embeddings adds a second pass, and hessian adds an energy-only pass plus three backward passes, so a Hessian label costs roughly three to four times an energy-and-force label. Each redrawn probe is a new objective. Leave probe_seed unset wherever coverage of the Hessian comes from redrawing, as in training and offline labeling. Pin it wherever a number is compared across passes, as DistillationStrategy does per validation batch through the call-time probe_seed of label().

label(batch, *, probe_seed=None)[source]#

Return the requested teacher signals for batch.

Parameters:
  • batch (Batch) – Batch to score. Restored to its incoming state before returning, including neighbor tensors, any pre-existing embeddings, and any field the teacher writes while scoring.

  • probe_seed (int | None, optional) – Seed of this labeling’s hessian probe direction, in place of the scorer’s own probe_seed, which is left as it is. Default None uses the scorer’s setting.

Returns:

Mapping from batch field name to (detached tensor, level).

Return type:

TeacherLabels

Raises:
  • RuntimeError – If the teacher omits an output or embedding a requested signal needs.

  • ValueError – If neighbor_list="reuse" and batch carries no list the teacher’s format reads, or one stamped with another cutoff.

label_hvp(batch, probe)[source]#

Return the teacher’s Hessian-vector product along probe.

The teacher’s energy is differentiated twice with respect to the positions of batch. The pass is narrowed to the energy and runs under the same autocast setting and the same neighbor-list and field isolation as label(). The batch is left as it was found, including a composed teacher’s wired intermediates and swapped autograd leaves.

Parameters:
  • batch (Batch) – Batch to differentiate the teacher’s energy on.

  • probe (NodePositions) – Probe direction of shape (V, 3), matching the batch’s positions.

Returns:

Detached Hessian-vector product of shape (V, 3), cast to dtype when one is configured.

Return type:

Forces

Raises:

RuntimeError – If the teacher returns no energy, or is not twice differentiable with respect to positions.

Examples

>>> import torch
>>> from nvalchemi.training.distillation import InProcessTeacherScorer
>>> scorer = InProcessTeacherScorer(teacher, ["hessian"])
>>> probe = torch.randn_like(batch.positions)
>>> scorer.label_hvp(batch, probe).shape
torch.Size([12, 3])

Notes

One product costs one forward and the three backward passes HessianOperator takes: the first position derivative, its connectivity probe, and the product. A Hutchinson average over k probes takes k calls. That average is left to the caller, because the loss consumes one materialized target per batch.