nvalchemi.training.distillation.InProcessTeacherScorer#
- class nvalchemi.training.distillation.InProcessTeacherScorer(teacher, signals, *, dtype=None, neighbor_list='rebuild', autocast=False, probe_seed=None)[source]#
Score a batch with a teacher model loaded in the current process.
The scorer owns the teacher’s evaluation contract: it narrows
active_outputsto the outputs the requested signals need, builds and afterwards restores whatever neighbor list the teacher requires — or, on request, consumes the one the batch already carries — picks the grad mode the teacher’s autograd outputs need, runs the pass under itsautocastsetting, detaches every result, and normalizes each signal to its canonical shape. The batch is left exactly as it was found, so a scorer can be called mid-training on a live batch.Each signal is a
TeacherSignalmapping one teacher output to one batch field at one level. The built-in ones (BUILTIN_SIGNALS) are requested by name:energytoteacher_energy(B, 1)andstresstoteacher_stress(B, 3, 3)at system level;forcestoteacher_forces(V, 3),atomic_energiestoteacher_atomic_energies(V,),embeddings(fromcompute_embeddings()) toteacher_node_embeddings(V, D), andhessian(fromlabel_hvp()) toteacher_hvp(V, 3)at node level. Thehessiansignal is the teacher’s Hessian-vector product, the product of its energy Hessian with a random probe direction; that direction is written toteacher_hvp_probe. Any other teacher output is requested as aTeacherSignalof its own. The resolved specs are published assignal_specsand the fields they write aslabel_fields.- Parameters:
teacher (BaseModelMixin) – Model wrapper producing the signals. Placed in evaluation mode at construction and for the duration of every
label()call, with the mode it arrived in restored afterwards; its parameters and theirrequires_gradflags are never modified.signals (Iterable[str | TeacherSignal]) – Signals to produce: built-in names or
TeacherSignalspecs, each naming an output in the teacher’s declaredoutputs.dtype (torch.dtype | None, optional) – Cast floating-point outputs to this dtype. Any floating-point dtype is accepted; whether a store can hold it is checked by
label_dataset(), and a labeled store reads back at the reading dataset’spositionsdtype regardless. DefaultNone(keep the teacher’s dtype).neighbor_list (NeighborListPolicy, optional) –
"rebuild"builds the teacher’s own neighbor list for every call, hiding whatever list the batch carries, and rolls it back afterwards."reuse"hands the teacher the batch’s list instead: the batch must carry the keys the teacher’s format reads, and a cutoff stamp, when the batch has one, must equal the teacher’s; otherwiselabel()raises rather than falling back. Whether the list holds each pair once or twice is not recorded on the batch, so a reused list must match the teacher’shalf_listby construction. Default"rebuild".autocast (bool | torch.dtype | None, optional) – Autocast mode for the scoring pass. The ambient autocast region is whatever autocast state is in force where
label()is called, such as a caller’s AMP region.Falsedisables autocast for the pass, so an ambient region never reaches the teacher.Noneleaves the ambient state untouched, so the teacher runs under the caller’s region when one is open.Trueor a floating-pointtorch.dtypeenables autocast for the pass, whether or not an ambient region is open: a dtype sets the autocast dtype, andTruekeeps the autocast dtype in force for the device, which is the device default when no region is open and the ambient region’s dtype when one is. dtype applies after this setting: the teacher produces each label at the precision this mode gives, and dtype, when set, then casts it. DefaultFalse.probe_seed (int | None, optional) – Seed of the generator the
hessianprobe direction is drawn from. DefaultNonedraws from the global RNG, a fresh direction per labeling. A call tolabel()can override it for that labeling alone; see the Notes.
- Raises:
ValueError – If signals is empty, names a signal that is neither built in nor a
TeacherSignal, gives two specs one name or one field, names a model output the teacher does not declare, gives a custom spec no model output, requests"embeddings"from a teacher that publishes no node-embedding shape, requests"hessian"from a teacher that declares noenergyoutput, dtype is not a floating-point dtype, neighbor_list is neither"rebuild"nor"reuse", autocast is neitherNone, a bool, nor a floating-point dtype, or teacher is a composition planning more than one neighbor-list source.
Examples
>>> from nvalchemi.training.distillation import InProcessTeacherScorer >>> scorer = InProcessTeacherScorer(teacher, ["energy", "forces"]) >>> labels = scorer.label(batch) >>> labels["teacher_forces"][1] 'node'
A teacher output the built-in table does not cover is labeled through a spec of its own:
>>> from nvalchemi.training.distillation import TeacherSignal >>> charges = TeacherSignal("charges", "charges", "teacher_charges", "node") >>> scorer = InProcessTeacherScorer(teacher, ["energy", charges]) >>> scorer.label_fields ('teacher_charges', 'teacher_energy')
Notes
Under the default
neighbor_list="rebuild"the teacher’s list is built for the forward pass and rolled back, and a list a composed pipeline keeps as an instance attribute, along with its captured per-source table, is hidden from the teacher for the whole of scoring."reuse"is for the case where the student has already built the list the teacher needs, in the teacher’s format and at its cutoff, and one build per step is one too many; the scorer then checks only what it cannot infer and refuses the batch by name when the list is missing or stamped with another cutoff. A teacher composition planning more than one neighbor-list source is refused at construction, because the scorer builds one list per batch; compose it to plan a single list instead (neighbor_adaptation="always"or a large enoughmax_cutoff_ratio).requires_gradonpositionsand the teacher’s autograd inputs is restored after each call, as is every field a composed teacher writes onto the batch to wire one stage into the next.Label precision is the scorer’s decision. Every consumer in the package calls
label()inside the ambient autocast region and opens none of its own, so the autocast setting decides which autocast mode the teacher runs under. The default disables autocast, so a label taken during a mixed-precision training or generation phase equals the onelabel_dataset()writes offline. Inside a caller’s region,Trueand a dtype differ:Trueenables autocast at the region’s dtype, whereas a dtype pins the pass to itself.Forward-pass signals share one teacher pass.
embeddingsadds a second pass, andhessianadds an energy-only pass plus three backward passes, so a Hessian label costs roughly three to four times an energy-and-force label. Each redrawn probe is a new objective. Leaveprobe_seedunset wherever coverage of the Hessian comes from redrawing, as in training and offline labeling. Pin it wherever a number is compared across passes, asDistillationStrategydoes per validation batch through the call-timeprobe_seedoflabel().- label(batch, *, probe_seed=None)[source]#
Return the requested teacher signals for batch.
- Parameters:
batch (Batch) – Batch to score. Restored to its incoming state before returning, including neighbor tensors, any pre-existing embeddings, and any field the teacher writes while scoring.
probe_seed (int | None, optional) – Seed of this labeling’s
hessianprobe direction, in place of the scorer’s ownprobe_seed, which is left as it is. DefaultNoneuses the scorer’s setting.
- Returns:
Mapping from batch field name to
(detached tensor, level).- Return type:
- Raises:
RuntimeError – If the teacher omits an output or embedding a requested signal needs.
ValueError – If
neighbor_list="reuse"and batch carries no list the teacher’s format reads, or one stamped with another cutoff.
- label_hvp(batch, probe)[source]#
Return the teacher’s Hessian-vector product along probe.
The teacher’s energy is differentiated twice with respect to the positions of batch. The pass is narrowed to the energy and runs under the same
autocastsetting and the same neighbor-list and field isolation aslabel(). The batch is left as it was found, including a composed teacher’s wired intermediates and swapped autograd leaves.- Parameters:
batch (Batch) – Batch to differentiate the teacher’s energy on.
probe (NodePositions) – Probe direction of shape
(V, 3), matching the batch’s positions.
- Returns:
Detached Hessian-vector product of shape
(V, 3), cast todtypewhen one is configured.- Return type:
Forces
- Raises:
RuntimeError – If the teacher returns no energy, or is not twice differentiable with respect to positions.
Examples
>>> import torch >>> from nvalchemi.training.distillation import InProcessTeacherScorer >>> scorer = InProcessTeacherScorer(teacher, ["hessian"]) >>> probe = torch.randn_like(batch.positions) >>> scorer.label_hvp(batch, probe).shape torch.Size([12, 3])
Notes
One product costs one forward and the three backward passes
HessianOperatortakes: the first position derivative, its connectivity probe, and the product. A Hutchinson average overkprobes takeskcalls. That average is left to the caller, because the loss consumes one materialized target per batch.