nvalchemi.training.distillation.evaluation.build_acceptance_report#

nvalchemi.training.distillation.evaluation.build_acceptance_report(evaluations, thresholds=None, *, bars=DEFAULT_BARS)[source]#

Turn a family of student evaluations into verdicts.

Parameters:
  • evaluations (Sequence[StudentEvaluation]) – One evaluation per candidate student. Names must be unique, since they key the report’s exports.

  • thresholds (AcceptanceThresholds | None, optional) – Bars to apply. Default None (no bars: every student is accepted and the report is a comparison table).

  • bars (Sequence[AcceptanceBar], optional) – Table of bars to apply. Extend DEFAULT_BARS to gate a measurement filed under StudentEvaluation.extra. Default DEFAULT_BARS.

Returns:

Verdicts aligned with evaluations.

Return type:

AcceptanceReport

Raises:

ValueError – If evaluations is empty, or two students share a name. If two entries of bars share a name. If thresholds.extra sets a limit for a bar that the table does not carry, or for a built-in bar that is set through its own field. If the students were not all scored on the same held-out set.

Examples

>>> from nvalchemi.training.distillation.evaluation import (
...     AcceptanceThresholds,
...     StudentEvaluation,
...     build_acceptance_report,
... )
>>> report = build_acceptance_report(
...     [StudentEvaluation(name="student-s", accuracy=metrics)],
...     AcceptanceThresholds(max_forces_mae=0.05),
... )
>>> report.accepted
True