nvalchemi.training.distillation.evaluation.measured_bars#

nvalchemi.training.distillation.evaluation.measured_bars(*families, accuracy_quantities=None, bars=DEFAULT_BARS)[source]#

Return the acceptance bars families hold enough measurements to decide.

A bar counts as measured only when every family it reads was supplied, because build_acceptance_report() fails a student on a bar whose measurement is missing rather than skipping it. max_from_scratch_ratio therefore needs both "accuracy" and "baseline_accuracy". accuracy_quantities narrows the answer further. An accuracy pass fills only the quantities it compared, so a held-out set scored on energy alone leaves max_forces_mae as unfillable as no pass at all. One bar has a precondition that no argument here can express, and it is reported on the strength of its family alone: max_energy_drift_per_atom_per_ns needs a StabilityMonitor built with timestep_fs. A check that fails for that reason says so.

Parameters:
  • *families (str) – Slots of a StudentEvaluation the caller fills, or "extra:<family>" for a number map under its extra. Naming none returns an empty set.

  • accuracy_quantities (Sequence[AccuracyQuantity] | None, optional) – Quantities the accuracy pass compared. Default None (every quantity).

  • bars (Sequence[AcceptanceBar], optional) – Table of bars to answer for. Default DEFAULT_BARS.

Returns:

Names of the bars that may be set: fields of AcceptanceThresholds, or keys of its extra.

Return type:

frozenset[str]

Raises:

ValueError – If a name in families is not a measurement family the table reads, or a name in accuracy_quantities is not an accuracy quantity.

Examples

>>> from nvalchemi.training.distillation.evaluation import measured_bars
>>> sorted(measured_bars("accuracy"))
['max_energy_per_atom_mae', 'max_forces_mae', 'max_stress_mae', 'min_force_cosine']
>>> sorted(measured_bars("accuracy", accuracy_quantities=["energy"]))
['max_energy_per_atom_mae']