nvalchemi.training.distillation.OnPolicyConfig#

class nvalchemi.training.distillation.OnPolicyConfig(*, replay_ratio, training_steps_per_segment, batch_size=8, generation_steps=100, label_frequency=100, replay_capacity=None, replay_eviction='fifo', replay_device=None, seed=0, rank_seed_stride=1000003, require_wrapped_student=True, fmax=None, max_trajectory_steps=None, quarantine_diverged=False, weight_sync_frequency=1, probe=True, samples_equilibrium=None, dynamics, teacher_scorer, initial_structures, replay_admission=None, capture_sink=None, convergence_hook=None, divergence=None)[source]#

Settings and live objects of one on-policy distillation segment loop.

Each segment has two phases. The generation phase runs the student’s own propagator for generation_steps steps and labels frames with the teacher as it goes. The training phase then takes training_steps_per_segment optimizer steps. Its batches are a mixture of the reference dataset and the replay buffer, split at replay_ratio. The propagator holds the module the trainer updates, so each segment generates from a fresher policy than the last. The scalar settings come from OnPolicySettings, which this class inherits so that a recipe stays flat. settings returns a detached copy of them for a pre-flight check or a restart bundle.

The propagator can be any BaseDynamics. A relaxation optimizer such as FIRE2 drives the loop exactly as a thermostat does. The initial structures must carry every field the propagator updates in place through __provides_keys__. For every shipped propagator these include velocities, and the variable-cell ones also need a cell. The model outputs named by __needs_keys__ are computed before the first step, so they need not be present. Construction checks one row, so a missing field is a construction error rather than a failure on the first step. The propagator’s compute() then runs once on that row. This rejects declarations that no longer match the implementation, such as a __needs_keys__ output the student never produces or a field compute() reads that nothing declared. The cost is one student forward pass at construction. A graph model is probed with the neighbor list its neighbor_config declares, built on the row and rolled back afterwards. A model that plans more than one neighbor-list source is not probed.

A relaxation propagator needs a trajectory lifecycle. Relaxations converge, and a converged structure that keeps being propagated fills the replay buffer with near-duplicates of a frame it already holds. fmax turns the lifecycle on. A converged structure freezes and is stored once, as the minimum it reached. At the segment boundary it graduates, that is, it leaves the batch, and the initial structures backfill a fresh structure in its place for as long as the source holds rows. An InitialStructures built with recycle=True wraps to its first row instead of letting the batch narrow. Generation ends when the last trajectory finishes, and the remaining training steps draw on the frames already in the buffer.

A thermostatted propagator needs the same lifecycle for a different reason: its trajectories never end, so the initial batch is the only set of structures the run ever propagates. max_trajectory_steps retires a trajectory after that many propagator steps, stored once like a converged one, and backfills its place from the initial structures, so the run rotates through the whole source. It works alone or beside fmax.

Parameters:
  • dynamics (BaseDynamics) – Propagator that generates on-policy frames. It holds the student module.

  • teacher_scorer (TeacherScorer) – Scorer that labels generated frames. A custom scorer that declares label_fields lets the fields it writes be known before the run.

  • initial_structures (InitialStructuresSource) – Structures the generated trajectories start from. They are served from one position that backfills and restarts share. A multi-rank launch deals them out strided, every world_size-th structure to each rank. Pass an InitialStructures, any other object that implements the protocol, or a bare dataset, which is wrapped in an InitialStructures.

  • capture_sink (DataSink | None, optional) – Sink that stages each segment’s labeled frames until the segment boundary drains them into the replay buffer. A GPUBuffer keeps the staged frames on the generation device and avoids a device-to-host copy per frame. Default None builds a host-memory sink per segment.

  • replay_eviction ({"fifo"} | EvictionPolicy, optional) – Eviction policy of the replay buffer: the name "fifo" or a live EvictionPolicy instance. Default "fifo".

  • replay_admission (AdmissionPolicy | None, optional) – Predicate that selects which captured frames each segment admits into the replay buffer. Default None admits every captured frame.

  • convergence_hook (ConvergenceHook | None, optional) – Live criterion that decides when a generated trajectory is finished, used instead of the fmax threshold. Default None.

  • divergence (Callable[[Batch], Bool[torch.Tensor, "G"]] | None, optional) – Predicate over the live frame that returns one boolean per graph, True where the trajectory diverged. The lifecycle freezes a flagged graph on the step it is flagged and keeps it out of both capture routes. At the segment boundary it retires the graph and backfills its place. Default None uses nonfinite_divergence().

  • replay_ratio (Annotated[float, Ge(ge=0.0), Le(le=1.0)])

  • training_steps_per_segment (Annotated[int, Gt(gt=0)])

  • batch_size (Annotated[int, Gt(gt=0)])

  • generation_steps (Annotated[int, Gt(gt=0)])

  • label_frequency (Annotated[int, Gt(gt=0)])

  • replay_capacity (Annotated[int | None, Gt(gt=0)])

  • replay_device (str | None)

  • seed (Annotated[int, Ge(ge=0)])

  • rank_seed_stride (Annotated[int, Gt(gt=0)])

  • require_wrapped_student (bool)

  • fmax (Annotated[float | None, Gt(gt=0)])

  • max_trajectory_steps (Annotated[int | None, Gt(gt=0)])

  • quarantine_diverged (bool)

  • weight_sync_frequency (Annotated[int, Gt(gt=0)])

  • probe (bool)

  • samples_equilibrium (bool | None)

Raises:

ValueError – If a setting is out of range, if both fmax and convergence_hook are set, if convergence_hook cannot manage the lifecycle, if initial_structures recycles while neither a criterion nor max_trajectory_steps is set, if a lifecycle is paired with a DomainParallel propagator or a multi-sub-stage FusedStage, if initial_structures is neither a source nor a dataset, if the initial structures lack a field the propagator needs for its first step, or if the propagator’s compute() on one row contradicts its declared keys.

Examples

>>> from nvalchemi.training.distillation import (
...     InProcessTeacherScorer,
...     OnPolicyConfig,
...     InitialStructures,
... )
>>> config = OnPolicyConfig(
...     dynamics=NVTLangevin(student, dt=0.5, temperature=300.0),
...     teacher_scorer=InProcessTeacherScorer(teacher, ["energy", "forces"]),
...     initial_structures=InitialStructures(dataset),
...     replay_ratio=0.25,
...     training_steps_per_segment=32,
...     batch_size=16,
...     generation_steps=50,
...     label_frequency=10,
...     replay_capacity=8192,
... )

The same loop over relaxation paths. Each structure graduates once its max force norm falls below 0.05, and the next initial structure takes its place:

>>> config = OnPolicyConfig(
...     dynamics=FIRE2(student, dt=0.05),
...     teacher_scorer=InProcessTeacherScorer(teacher, ["energy", "forces"]),
...     initial_structures=InitialStructures(dataset, recycle=True),
...     fmax=0.05,
...     replay_ratio=0.25,
...     training_steps_per_segment=32,
...     batch_size=16,
...     generation_steps=50,
...     label_frequency=10,
... )

Notes

Any TeacherScorer may drive generation. A custom scorer can declare label_fields. The declaration lets DistillationStrategy check the generated fields against reference_dataset at construction. It also keeps TeacherLabelHook from re-scoring a frame that is dispatched again. A custom teacher_* field the scorer writes is an ordinary loss target, so the reference dataset and any validation data must carry it too.

The loop sizes capture_sink. A segment captures at most one frame per trajectory per labeled step, including the forced last frame. The sink must therefore hold (generation_steps + 1) frames per trajectory in the propagated batch. A configured sink with less capacity is grown through resize(capacity) when it satisfies ResizableSink, and rejected otherwise. The sink must also be empty when a segment starts, because everything it holds is drained into the replay buffer as generated frames.

capture_sink is runtime-only, like dynamics and teacher_scorer, so no recipe names it. A recipe does not name a policy instance either. settings records a custom replay_eviction as "fifo" with a warning, and a config rebuilt from those settings evicts FIFO until the policy is supplied again.

fmax stays a plain number that a recipe can hold. convergence_criterion is the live criterion the lifecycle drives. For an fmax threshold, it is built once with from_fmax(), migrating status 0 to the propagator’s exit_status. The same object is returned on every read, because the lifecycle registers that one object and removes it by identity. A criterion that has to be a live hook goes to convergence_hook instead. Setting both fmax and convergence_hook is refused.

A convergence_hook must migrate status on every step, from the status 0 the run stamps its structures with. A hook that only reports convergence would freeze and graduate nothing. A hook that skips steps graduates a structure late, so both capture routes would store it. The construction probe also dispatches a copy of the criterion to the probed row. A criterion that raises on the propagator’s outputs, or whose firing leaves status unchanged, is therefore refused here. A criterion that reads a key no compute() produces is not dispatched, because a hook may write that key during the step; a warning names the key instead. probe=False skips this dispatch along with the forward pass it reads.

divergence ends a trajectory before it converges. It has the same shape as AdmissionPolicy: it takes the live frame and returns one boolean per graph. The default flags a graph whose positions or forces are no longer finite. Use a custom predicate for a student whose forces explode to finite but unphysical values, or for a criterion on the energy. Like replay_admission, the predicate is runtime-only. A predicate that returns anything but one boolean per graph is refused on its first dispatch, and the error names the shape it returned.

For the duration of the loop, the criterion also becomes the propagator’s convergence detector. It must be the only hook that migrates status, so a propagator that carries a second migrating ConvergenceHook is refused. A FusedStage builds such a hook for every sub-stage except the last, and for the last one whenever it declares a convergence_hook. Only a single-sub-stage fused stage without a criterion of its own is therefore accepted. A DomainParallel propagator is refused with a criterion as well: its step dispatches no ON_GRADUATE, the stage the converged route captures at, so its minima would never be stored. A migrator already on the propagator is refused when the config is built, and one registered afterwards is refused when the run starts. See Distillation API for the capture routes and the backfill.

Distribution-matching objectives are defined on equilibrium ensembles, and a relaxation path is not one. A BoltzmannMatchingLoss is therefore refused at construction beside a relaxation propagator or any convergence criterion, unless samples_equilibrium=True declares that the propagator does sample an equilibrium ensemble. False refuses the term whatever the propagator is. The term should also have a bounded replay_capacity. Energy, force, and atomic-energy matching are pointwise, so they distill a relaxation path exactly as they distill a trajectory.

property convergence_criterion: ConvergenceHook | None#

Return the criterion the trajectory lifecycle drives, or None.

A convergence_hook is returned as is. An fmax threshold is turned into a criterion on first read, migrating status 0 to the propagator’s exit_status. The same object is returned for the life of the config, because the lifecycle registers it on the propagator and removes it again by identity.

Returns:

The live criterion, or None for a run managing no lifecycle.

Return type:

ConvergenceHook | None

property manages_lifecycle: bool#

Whether the segment loop retires and backfills trajectories.

Returns:

True when a convergence criterion (fmax or convergence_hook) or a max_trajectory_steps lifetime is set; False for a run that propagates its initial batch for the whole run.

Return type:

bool

model_config = {'arbitrary_types_allowed': True, 'extra': 'forbid'}#

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context, /)#

This function is meant to behave like a BaseModel method to initialise private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:
  • self (BaseModel) – The BaseModel instance.

  • context (Any) – The context.

Return type:

None

property settings: OnPolicySettings#

Detached copy of the declarative settings, for a recipe or a restart bundle.

Returns:

The scalar settings of this config, validated on their own. The copy holds no reference to the live objects.

Return type:

OnPolicySettings

Warns:

UserWarning – If replay_eviction is a policy instance other than FIFO. The copy records it as "fifo".