nvalchemi.training.distillation.evaluation.StudentEvaluation#
- class nvalchemi.training.distillation.evaluation.StudentEvaluation(*, name, accuracy, stability=None, extensivity=None, rdf=None, baseline_accuracy=None, num_parameters=None, weights=None, extra=<factory>)[source]#
Everything measured about one candidate student.
Only name and accuracy are required. Each remaining slot holds a measurement that a caller may or may not have run. A bar aimed at an empty slot fails the student rather than passing it silently. weights is not a slot: it notes where the numbers came from, and no bar reads it.
- name#
Label the student is reported under.
- Type:
str
- accuracy#
Held-out errors, from
evaluate_accuracy().- Type:
- stability#
Trajectory conservation metrics, from
metrics().- Type:
StabilityMetrics | None
- extensivity#
Energy-scaling error, from
extensivity_error().- Type:
ExtensivityMetrics | None
- rdf#
Structural match against a reference trajectory, from
compare_radial_distributions().- Type:
RDFComparison | None
- baseline_accuracy#
The same accuracy evaluation run on an equal-size student trained from scratch. The from-scratch bar compares the student against it. A baseline whose graph and atom counts differ from those of accuracy fails the bar rather than being used for a ratio.
- Type:
AccuracyMetrics | None
- num_parameters#
Parameter count, reported beside the student’s errors.
- Type:
int | None
- weights#
Which of the student’s weights the numbers were measured on: its EMA-averaged weights (
"ema") or its live ones ("raw"). Only the caller knows whether it passedevaluate_accuracy()astrategy.inference_model, so record it here.Nonerecords nothing, which is not the same as"raw".- Type:
Literal[“ema”, “raw”] | None
- extra#
Measurements outside the typed slots, as one flat map of numbers per family. A custom
AcceptanceBarreads a family through"extra:<family>". Default empty.- Type:
Mapping[str, Mapping[str, float]]
- Raises:
TypeError – If a measurement slot holds anything but its own metrics class.
ValueError – If weights names neither of the two weight sets.
- Parameters:
name (str)
accuracy (AccuracyMetrics)
stability (StabilityMetrics | None)
extensivity (ExtensivityMetrics | None)
rdf (RDFComparison | None)
baseline_accuracy (AccuracyMetrics | None)
num_parameters (int | None)
weights (Literal['ema', 'raw'] | None)
extra (dict[str, dict[str, float]])
- classmethod from_dict(data)[source]#
Rebuild an evaluation, and its measurements, from a
to_dict()export.An entry taken straight out of
AcceptanceReport.to_dict()is also accepted. Itsverdictis dropped, because verdicts are formed from the thresholds of the report being built, not carried between jobs.- Raises:
ValueError – If the export, or one of its nested measurements, carries a key the record does not declare or omits a required one.
- Parameters:
data (Mapping[str, Any])
- Return type:
- model_config = {'extra': 'forbid', 'frozen': True}#
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].