nvalchemi.training.distillation.OnPolicyConfig#
- class nvalchemi.training.distillation.OnPolicyConfig(*, replay_ratio, training_steps_per_segment, batch_size=8, generation_steps=100, label_frequency=100, replay_capacity=None, replay_eviction='fifo', replay_device=None, seed=0, rank_seed_stride=1000003, require_wrapped_student=True, fmax=None, max_trajectory_steps=None, quarantine_diverged=False, weight_sync_frequency=1, probe=True, samples_equilibrium=None, dynamics, teacher_scorer, initial_structures, replay_admission=None, capture_sink=None, convergence_hook=None, divergence=None)[source]#
Settings and live objects of one on-policy distillation segment loop.
Each segment has two phases. The generation phase runs the student’s own propagator for
generation_stepssteps and labels frames with the teacher as it goes. The training phase then takestraining_steps_per_segmentoptimizer steps. Its batches are a mixture of the reference dataset and the replay buffer, split atreplay_ratio. The propagator holds the module the trainer updates, so each segment generates from a fresher policy than the last. The scalar settings come fromOnPolicySettings, which this class inherits so that a recipe stays flat.settingsreturns a detached copy of them for a pre-flight check or a restart bundle.The propagator can be any
BaseDynamics. A relaxation optimizer such asFIRE2drives the loop exactly as a thermostat does. The initial structures must carry every field the propagator updates in place through__provides_keys__. For every shipped propagator these includevelocities, and the variable-cell ones also need acell. The model outputs named by__needs_keys__are computed before the first step, so they need not be present. Construction checks one row, so a missing field is a construction error rather than a failure on the first step. The propagator’scompute()then runs once on that row. This rejects declarations that no longer match the implementation, such as a__needs_keys__output the student never produces or a fieldcompute()reads that nothing declared. The cost is one student forward pass at construction. A graph model is probed with the neighbor list itsneighbor_configdeclares, built on the row and rolled back afterwards. A model that plans more than one neighbor-list source is not probed.A relaxation propagator needs a trajectory lifecycle. Relaxations converge, and a converged structure that keeps being propagated fills the replay buffer with near-duplicates of a frame it already holds.
fmaxturns the lifecycle on. A converged structure freezes and is stored once, as the minimum it reached. At the segment boundary it graduates, that is, it leaves the batch, and the initial structures backfill a fresh structure in its place for as long as the source holds rows. AnInitialStructuresbuilt withrecycle=Truewraps to its first row instead of letting the batch narrow. Generation ends when the last trajectory finishes, and the remaining training steps draw on the frames already in the buffer.A thermostatted propagator needs the same lifecycle for a different reason: its trajectories never end, so the initial batch is the only set of structures the run ever propagates.
max_trajectory_stepsretires a trajectory after that many propagator steps, stored once like a converged one, and backfills its place from the initial structures, so the run rotates through the whole source. It works alone or besidefmax.- Parameters:
dynamics (BaseDynamics) – Propagator that generates on-policy frames. It holds the student module.
teacher_scorer (TeacherScorer) – Scorer that labels generated frames. A custom scorer that declares
label_fieldslets the fields it writes be known before the run.initial_structures (InitialStructuresSource) – Structures the generated trajectories start from. They are served from one position that backfills and restarts share. A multi-rank launch deals them out strided, every
world_size-th structure to each rank. Pass anInitialStructures, any other object that implements the protocol, or a bare dataset, which is wrapped in anInitialStructures.capture_sink (DataSink | None, optional) – Sink that stages each segment’s labeled frames until the segment boundary drains them into the replay buffer. A
GPUBufferkeeps the staged frames on the generation device and avoids a device-to-host copy per frame. DefaultNonebuilds a host-memory sink per segment.replay_eviction ({"fifo"} | EvictionPolicy, optional) – Eviction policy of the replay buffer: the name
"fifo"or a liveEvictionPolicyinstance. Default"fifo".replay_admission (AdmissionPolicy | None, optional) – Predicate that selects which captured frames each segment admits into the replay buffer. Default
Noneadmits every captured frame.convergence_hook (ConvergenceHook | None, optional) – Live criterion that decides when a generated trajectory is finished, used instead of the
fmaxthreshold. DefaultNone.divergence (Callable[[Batch], Bool[torch.Tensor, "G"]] | None, optional) – Predicate over the live frame that returns one boolean per graph,
Truewhere the trajectory diverged. The lifecycle freezes a flagged graph on the step it is flagged and keeps it out of both capture routes. At the segment boundary it retires the graph and backfills its place. DefaultNoneusesnonfinite_divergence().replay_ratio (Annotated[float, Ge(ge=0.0), Le(le=1.0)])
training_steps_per_segment (Annotated[int, Gt(gt=0)])
batch_size (Annotated[int, Gt(gt=0)])
generation_steps (Annotated[int, Gt(gt=0)])
label_frequency (Annotated[int, Gt(gt=0)])
replay_capacity (Annotated[int | None, Gt(gt=0)])
replay_device (str | None)
seed (Annotated[int, Ge(ge=0)])
rank_seed_stride (Annotated[int, Gt(gt=0)])
require_wrapped_student (bool)
fmax (Annotated[float | None, Gt(gt=0)])
max_trajectory_steps (Annotated[int | None, Gt(gt=0)])
quarantine_diverged (bool)
weight_sync_frequency (Annotated[int, Gt(gt=0)])
probe (bool)
samples_equilibrium (bool | None)
- Raises:
ValueError – If a setting is out of range, if both
fmaxandconvergence_hookare set, ifconvergence_hookcannot manage the lifecycle, ifinitial_structuresrecycles while neither a criterion normax_trajectory_stepsis set, if a lifecycle is paired with aDomainParallelpropagator or a multi-sub-stageFusedStage, ifinitial_structuresis neither a source nor a dataset, if the initial structures lack a field the propagator needs for its first step, or if the propagator’scompute()on one row contradicts its declared keys.
Examples
>>> from nvalchemi.training.distillation import ( ... InProcessTeacherScorer, ... OnPolicyConfig, ... InitialStructures, ... ) >>> config = OnPolicyConfig( ... dynamics=NVTLangevin(student, dt=0.5, temperature=300.0), ... teacher_scorer=InProcessTeacherScorer(teacher, ["energy", "forces"]), ... initial_structures=InitialStructures(dataset), ... replay_ratio=0.25, ... training_steps_per_segment=32, ... batch_size=16, ... generation_steps=50, ... label_frequency=10, ... replay_capacity=8192, ... )
The same loop over relaxation paths. Each structure graduates once its max force norm falls below
0.05, and the next initial structure takes its place:>>> config = OnPolicyConfig( ... dynamics=FIRE2(student, dt=0.05), ... teacher_scorer=InProcessTeacherScorer(teacher, ["energy", "forces"]), ... initial_structures=InitialStructures(dataset, recycle=True), ... fmax=0.05, ... replay_ratio=0.25, ... training_steps_per_segment=32, ... batch_size=16, ... generation_steps=50, ... label_frequency=10, ... )
Notes
Any
TeacherScorermay drive generation. A custom scorer can declarelabel_fields. The declaration letsDistillationStrategycheck the generated fields againstreference_datasetat construction. It also keepsTeacherLabelHookfrom re-scoring a frame that is dispatched again. A customteacher_*field the scorer writes is an ordinary loss target, so the reference dataset and any validation data must carry it too.The loop sizes
capture_sink. A segment captures at most one frame per trajectory per labeled step, including the forced last frame. The sink must therefore hold(generation_steps + 1)frames per trajectory in the propagated batch. A configured sink with less capacity is grown throughresize(capacity)when it satisfiesResizableSink, and rejected otherwise. The sink must also be empty when a segment starts, because everything it holds is drained into the replay buffer as generated frames.capture_sinkis runtime-only, likedynamicsandteacher_scorer, so no recipe names it. A recipe does not name a policy instance either.settingsrecords a customreplay_evictionas"fifo"with a warning, and a config rebuilt from those settings evicts FIFO until the policy is supplied again.fmaxstays a plain number that a recipe can hold.convergence_criterionis the live criterion the lifecycle drives. For anfmaxthreshold, it is built once withfrom_fmax(), migrating status0to the propagator’sexit_status. The same object is returned on every read, because the lifecycle registers that one object and removes it by identity. A criterion that has to be a live hook goes toconvergence_hookinstead. Setting bothfmaxandconvergence_hookis refused.A
convergence_hookmust migrate status on every step, from the status0the run stamps its structures with. A hook that only reports convergence would freeze and graduate nothing. A hook that skips steps graduates a structure late, so both capture routes would store it. The construction probe also dispatches a copy of the criterion to the probed row. A criterion that raises on the propagator’s outputs, or whose firing leavesstatusunchanged, is therefore refused here. A criterion that reads a key nocompute()produces is not dispatched, because a hook may write that key during the step; a warning names the key instead.probe=Falseskips this dispatch along with the forward pass it reads.divergenceends a trajectory before it converges. It has the same shape asAdmissionPolicy: it takes the live frame and returns one boolean per graph. The default flags a graph whose positions or forces are no longer finite. Use a custom predicate for a student whose forces explode to finite but unphysical values, or for a criterion on the energy. Likereplay_admission, the predicate is runtime-only. A predicate that returns anything but one boolean per graph is refused on its first dispatch, and the error names the shape it returned.For the duration of the loop, the criterion also becomes the propagator’s convergence detector. It must be the only hook that migrates status, so a propagator that carries a second migrating
ConvergenceHookis refused. AFusedStagebuilds such a hook for every sub-stage except the last, and for the last one whenever it declares aconvergence_hook. Only a single-sub-stage fused stage without a criterion of its own is therefore accepted. ADomainParallelpropagator is refused with a criterion as well: its step dispatches noON_GRADUATE, the stage the converged route captures at, so its minima would never be stored. A migrator already on the propagator is refused when the config is built, and one registered afterwards is refused when the run starts. See Distillation API for the capture routes and the backfill.Distribution-matching objectives are defined on equilibrium ensembles, and a relaxation path is not one. A
BoltzmannMatchingLossis therefore refused at construction beside a relaxation propagator or any convergence criterion, unlesssamples_equilibrium=Truedeclares that the propagator does sample an equilibrium ensemble.Falserefuses the term whatever the propagator is. The term should also have a boundedreplay_capacity. Energy, force, and atomic-energy matching are pointwise, so they distill a relaxation path exactly as they distill a trajectory.- property convergence_criterion: ConvergenceHook | None#
Return the criterion the trajectory lifecycle drives, or
None.A
convergence_hookis returned as is. Anfmaxthreshold is turned into a criterion on first read, migrating status0to the propagator’sexit_status. The same object is returned for the life of the config, because the lifecycle registers it on the propagator and removes it again by identity.- Returns:
The live criterion, or
Nonefor a run managing no lifecycle.- Return type:
ConvergenceHook | None
- property manages_lifecycle: bool#
Whether the segment loop retires and backfills trajectories.
- Returns:
Truewhen a convergence criterion (fmaxorconvergence_hook) or amax_trajectory_stepslifetime is set;Falsefor a run that propagates its initial batch for the whole run.- Return type:
bool
- model_config = {'arbitrary_types_allowed': True, 'extra': 'forbid'}#
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_post_init(context, /)#
This function is meant to behave like a BaseModel method to initialise private attributes.
It takes context as an argument since that’s what pydantic-core passes when calling it.
- Parameters:
self (BaseModel) – The BaseModel instance.
context (Any) – The context.
- Return type:
None
- property settings: OnPolicySettings#
Detached copy of the declarative settings, for a recipe or a restart bundle.
- Returns:
The scalar settings of this config, validated on their own. The copy holds no reference to the live objects.
- Return type:
- Warns:
UserWarning – If
replay_evictionis a policy instance other thanFIFO. The copy records it as"fifo".