nvalchemi.training.distillation.evaluation.measured_bars#
- nvalchemi.training.distillation.evaluation.measured_bars(*families, accuracy_quantities=None, bars=DEFAULT_BARS)[source]#
Return the acceptance bars families hold enough measurements to decide.
A bar counts as measured only when every family it reads was supplied, because
build_acceptance_report()fails a student on a bar whose measurement is missing rather than skipping it.max_from_scratch_ratiotherefore needs both"accuracy"and"baseline_accuracy". accuracy_quantities narrows the answer further. An accuracy pass fills only the quantities it compared, so a held-out set scored on energy alone leavesmax_forces_maeas unfillable as no pass at all. One bar has a precondition that no argument here can express, and it is reported on the strength of its family alone:max_energy_drift_per_atom_per_nsneeds aStabilityMonitorbuilt withtimestep_fs. A check that fails for that reason says so.- Parameters:
*families (str) – Slots of a
StudentEvaluationthe caller fills, or"extra:<family>"for a number map under itsextra. Naming none returns an empty set.accuracy_quantities (Sequence[AccuracyQuantity] | None, optional) – Quantities the accuracy pass compared. Default
None(every quantity).bars (Sequence[AcceptanceBar], optional) – Table of bars to answer for. Default
DEFAULT_BARS.
- Returns:
Names of the bars that may be set: fields of
AcceptanceThresholds, or keys of itsextra.- Return type:
frozenset[str]
- Raises:
ValueError – If a name in families is not a measurement family the table reads, or a name in accuracy_quantities is not an accuracy quantity.
Examples
>>> from nvalchemi.training.distillation.evaluation import measured_bars >>> sorted(measured_bars("accuracy")) ['max_energy_per_atom_mae', 'max_forces_mae', 'max_stress_mae', 'min_force_cosine'] >>> sorted(measured_bars("accuracy", accuracy_quantities=["energy"])) ['max_energy_per_atom_mae']