nvalchemi.training.distillation.evaluation.build_acceptance_report#
- nvalchemi.training.distillation.evaluation.build_acceptance_report(evaluations, thresholds=None, *, bars=DEFAULT_BARS)[source]#
Turn a family of student evaluations into verdicts.
- Parameters:
evaluations (Sequence[StudentEvaluation]) – One evaluation per candidate student. Names must be unique, since they key the report’s exports.
thresholds (AcceptanceThresholds | None, optional) – Bars to apply. Default
None(no bars: every student is accepted and the report is a comparison table).bars (Sequence[AcceptanceBar], optional) – Table of bars to apply. Extend
DEFAULT_BARSto gate a measurement filed underStudentEvaluation.extra. DefaultDEFAULT_BARS.
- Returns:
Verdicts aligned with evaluations.
- Return type:
- Raises:
ValueError – If evaluations is empty, or two students share a name. If two entries of bars share a name. If
thresholds.extrasets a limit for a bar that the table does not carry, or for a built-in bar that is set through its own field. If the students were not all scored on the same held-out set.
Examples
>>> from nvalchemi.training.distillation.evaluation import ( ... AcceptanceThresholds, ... StudentEvaluation, ... build_acceptance_report, ... ) >>> report = build_acceptance_report( ... [StudentEvaluation(name="student-s", accuracy=metrics)], ... AcceptanceThresholds(max_forces_mae=0.05), ... ) >>> report.accepted True