nvalchemi.training.distillation.ReplayBuffer#

class nvalchemi.training.distillation.ReplayBuffer(*, capacity=None, eviction='fifo', admission=None, device=None)[source]#

Hold generated frames for replay under one frozen key schema.

The buffer wraps an InMemoryDataset that grows one segment at a time, so a loader or a MultiDataset consumes it like any dataset. The first extend() freezes the incoming schema, including levels and dtypes, and every later call must match it exactly. append() keeps only the keys both sides hold and casts what it keeps to the resident dtypes. Without the frozen schema, one unlabeled frame would strip teacher_* from every frame already stored, and one arriving in a wider dtype would have its labels rounded to the resident dtype unreported.

A stored frame is a training sample rather than a propagator state. It holds the structure and its teacher_* labels, and none of the predictions the propagator wrote. TeacherLabelHook delivers frames in this shape, and build_mixed_loader() requires the reference dataset to match it. Two decisions are left to policy: what enters the buffer and what leaves it. An AdmissionPolicy masks the incoming frames before the schema check. When the buffer is over capacity, an EvictionPolicy names the frames to drop. The default is FIFO, which drops the oldest frames first.

Parameters:
  • capacity (int | None, optional) – Maximum number of frames kept. Bound it on long runs. Also bound it on any run whose objective reads a batch as a sample of the current policy, because a draw over a buffer that never retires frames is a draw over every policy the run has had. Default None (unbounded).

  • eviction ({"fifo"} | EvictionPolicy, optional) – Policy deciding which frames leave a full buffer. Default "fifo", which builds FIFO.

  • admission (AdmissionPolicy | None, optional) – Predicate that selects which frames each extend() admits. Default None admits every frame.

  • device (torch.device | str | None, optional) – Device the buffer keeps frames on and emits them from. A segment loop sets it from OnPolicyConfig.replay_device. Default None adopts the device of the first frames passed to extend() and moves every later frame to that device, so two capture routes on two devices can fill one buffer.

Raises:
  • ValueError – If capacity is not positive, or if eviction is a string other than "fifo".

  • TypeError – If eviction is neither that string nor an object with select, or if admission is not callable.

Examples

>>> from nvalchemi.training.distillation import ReplayBuffer
>>> buffer = ReplayBuffer(capacity=4096)
>>> buffer.extend(labeled_frames)
>>> len(buffer)
128

Notes

Frames are owned, not aliased. The buffer copies the batch that first fills it and concatenates later batches into fresh tensors. A propagator may therefore keep integrating the batch it handed over.

property dataset: InMemoryDataset#

Dataset view of the stored frames, for a loader to draw from.

extend(frames)[source]#

Admit frames into the buffer and evict down to capacity.

Runs without autograd, so a stored frame is always detached: a frame handed over with requires_grad set, or still attached to the graph that produced it, never links the resident frames to a graph that a training backward would free.

Parameters:

frames (Batch) – Frames to store, one graph each. The admission policy filters them first. The first call that admits frames freezes the buffer’s key schema, and later calls must match it. When no device was given at construction, that call also fixes the buffer’s device.

Raises:

ValueError – If the key schema or the field dtypes of the admitted frames differ from the buffer’s, if the admission policy returns anything but one boolean per graph, or if the eviction policy returns non-integer indices, an index outside the resident frames, or fewer frames than the buffer’s excess over capacity. A refused eviction leaves the buffer as it was before the call.

Return type:

None

property schema: frozenset[str]#

Frozen level.field schema every frame must match; empty until filled.