nvalchemi.training.distillation.OnPolicySettings#

class nvalchemi.training.distillation.OnPolicySettings(*, replay_ratio, training_steps_per_segment, batch_size=8, generation_steps=100, label_frequency=100, replay_capacity=None, replay_eviction='fifo', replay_device=None, seed=0, rank_seed_stride=1000003, require_wrapped_student=True, fmax=None, max_trajectory_steps=None, quarantine_diverged=False, weight_sync_frequency=1, probe=True, samples_equilibrium=None)[source]#

Declarative settings of one on-policy distillation segment loop.

Every field is a JSON scalar, so the settings validate without a propagator, a teacher, or a store. A recipe with invalid settings is therefore rejected before any teacher is loaded. OnPolicyConfig inherits these settings and adds the live objects the loop drives. That class and the strategy check whether those objects work with the loop.

Parameters:
  • replay_ratio (float) – Fraction of every training batch drawn from the replay buffer. The reference dataset supplies the rest.

  • training_steps_per_segment (int) – Optimizer steps taken per segment, one per training batch.

  • batch_size (int, optional) – Samples per training batch, counting both sources of the mixture. Default 8.

  • generation_steps (int, optional) – Propagator steps generated per segment. Default 100.

  • label_frequency (int, optional) – Propagator steps between teacher labelings. Each segment’s last frame is labeled in addition. Default 100.

  • replay_capacity (int | None, optional) – Frame capacity of the replay buffer. Default None (unbounded). A Boltzmann objective needs a bound; see the OnPolicyConfig Notes.

  • replay_eviction ({"fifo"}, optional) – Eviction policy of the replay buffer, as a name a recipe can store. A policy instance is passed to OnPolicyConfig instead. Default "fifo".

  • replay_device (str | None, optional) – Device the replay buffer keeps frames on. An index-less cuda means this rank’s current CUDA device. Default None uses the device the reference dataset emits its batches on, or host memory when there is no reference dataset.

  • seed (int, optional) – Base seed of every segment’s mixture sampler. Default 0.

  • rank_seed_stride (int, optional) – Seed offset between neighboring ranks on a multi-rank launch. Rank r adds r * rank_seed_stride to its seeds. Default 1_000_003.

  • require_wrapped_student (bool, optional) – Whether a multi-rank run refuses to start unless the SETUP stage replaced the student with a wrapper that owns it, as a DDPHook does. Default True.

  • samples_equilibrium (bool | None, optional) – Whether the propagator samples an equilibrium ensemble, the ensemble a distribution-matching objective is defined on. Default None infers the answer from the propagator: a stage whose class declares samples_equilibrium false, as the relaxation optimizers do, or a convergence criterion counts as not sampling one. True or False overrides that inference for the objective’s guard.

  • fmax (float | None, optional) – Max force norm below which a generated trajectory counts as finished. Setting it turns on the trajectory lifecycle of a relaxation run, in which converged structures leave the batch and fresh initial structures replace them. Default None ends no trajectory, which suits a molecular-dynamics run.

  • max_trajectory_steps (int | None, optional) – Propagator steps a generated trajectory lives for. A trajectory that reaches this age is frozen on that step, stored once, and retired at the segment boundary like a converged one, so a fresh initial structure takes its place. It turns on the trajectory lifecycle on its own, which is how a thermostatted run rotates through its initial structures instead of propagating the first batch forever. A multiple of generation_steps retires on segment boundaries, so no frozen graph idles inside a segment. Default None lets a trajectory live for the whole run.

  • quarantine_diverged (bool, optional) – Sanitize a diverged graph on the step it is flagged: its velocities and forces are zeroed and its positions made finite, so the frozen graph presents a finite geometry to the model on every remaining step of the segment. The propagator still evaluates the whole batch, frozen graphs included, and a non-finite or runaway geometry can break a model’s neighbor-list construction. The sanitized frame is never captured. Default False leaves the flagged state as it is.

  • weight_sync_frequency (int, optional) – Segments between weight syncs to the propagator. Default 1, the only accepted value while the propagator shares the student module.

  • probe (bool, optional) – Whether OnPolicyConfig runs the propagator’s compute() on one initial structure at construction to check it against its declared keys. False defers any mismatch to the first step. Default True.

Raises:

ValueError – If a count or the threshold is not positive, if replay_ratio falls outside [0, 1] or is exactly 0, if rounding the ratio against the batch size gives one mixture source no sample in any batch, or if weight_sync_frequency is not 1.

Examples

>>> from nvalchemi.training.distillation import OnPolicySettings
>>> settings = OnPolicySettings(replay_ratio=0.25, training_steps_per_segment=32)
>>> settings.batch_size
8

Notes

label_frequency is the throughput setting. It counts against the propagator’s cumulative step_count, so the labeling cadence does not restart at a segment boundary. Each segment also labels the frame it ends on, and the cadence dispatch adjacent to that forced label is skipped. With generation_steps a multiple of label_frequency, each trajectory is therefore labeled generation_steps // label_frequency times per segment, which is once per segment only when the two are equal. The first segment pays one more, for the cadence dispatch at step 0. training_steps_per_segment counts training batches. It equals the number of optimizer steps only while every batch takes one step. fmax is compared against the student’s forces, which are the forces the propagator follows. It therefore measures the convergence of the relaxation itself.

Size replay_capacity as a multiple of the trajectory count. Otherwise FIFO eviction removes only part of one labeled step’s frames, which over-represents the structures at the back of the batch. Space the seed of replicate runs by at least num_steps // training_steps_per_segment, because the sampler adds the segment index to it. See Distillation API.

On a multi-rank launch, each rank adds rank * rank_seed_stride to seed and, through seed_offset(), to every random_seed that dynamics and its sub-stages expose. Ranks therefore draw from the reference dataset independently and apply different thermostat noise to the structures they were dealt. Both streams add a step counter to their seed, so the stride has to stay above every counter the run reaches; the default, 1_000_003, does so for a run whose counters stay below it. Set a different stride when a replicate launch’s seeds would land on another rank’s stride. A stage that holds randomness the offset cannot move, such as a torch.Generator with no integer random_seed, is named in a warning, and the caller must give it a rank-distinct seed.

A multi-rank run also checks that the SETUP stage replaced the student with a gradient-synchronizing wrapper. A wrapper that works in place, such as FSDP2’s fully_shard or hook-based synchronization, leaves the student object unchanged and fails that check. require_wrapped_student=False waives the check for such a wrapper, and keeping the ranks’ students in step then becomes the caller’s responsibility.

model_config = {'extra': 'forbid'}#

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].