nvalchemi.training.distillation.evaluation.AccuracyQuantitySpec#

class nvalchemi.training.distillation.evaluation.AccuracyQuantitySpec(name, prediction_key, reference_key, signal=None, supervised_loss=None)[source]#

Where one accuracy quantity’s prediction and target are read from.

The built-in quantities are BUILTIN_ACCURACY_QUANTITIES. A custom quantity is passed to evaluate_accuracy() as an instance, and its errors are reported in AccuracyMetrics.errors under its name. It can be a head that the training function publishes under its own key, or a built-in quantity read from another field.

Parameters:
  • name (str) – Name the quantity is requested and reported under.

  • prediction_key (str) – Key of the validation function’s output holding the prediction.

  • reference_key (str) – Batch field holding the dataset’s own label.

  • signal (str | None, optional) – Teacher signal the quantity is compared against when targets="teacher". The signal is resolved to its batch field through signal_fields(). Default None (no teacher target; name a teacher field in target_keys to compare against a teacher).

  • supervised_loss (Callable[..., BaseLossFunction] | None, optional) – Factory, called as supervised_loss(target_key=...), that builds the loss term driving the validation pass. Default None (the quantity is a diagnostic that never enters the pass’s loss).

Examples

>>> from nvalchemi.training.distillation.evaluation import (
...     AccuracyQuantitySpec,
... )
>>> charges = AccuracyQuantitySpec(
...     "charges", prediction_key="predicted_charges", reference_key="charges"
... )
>>> charges.supervised_loss is None
True