nvalchemi.training.distillation.evaluation.AccuracyQuantitySpec#
- class nvalchemi.training.distillation.evaluation.AccuracyQuantitySpec(name, prediction_key, reference_key, signal=None, supervised_loss=None)[source]#
Where one accuracy quantity’s prediction and target are read from.
The built-in quantities are
BUILTIN_ACCURACY_QUANTITIES. A custom quantity is passed toevaluate_accuracy()as an instance, and its errors are reported inAccuracyMetrics.errorsunder its name. It can be a head that the training function publishes under its own key, or a built-in quantity read from another field.- Parameters:
name (str) – Name the quantity is requested and reported under.
prediction_key (str) – Key of the validation function’s output holding the prediction.
reference_key (str) – Batch field holding the dataset’s own label.
signal (str | None, optional) – Teacher signal the quantity is compared against when
targets="teacher". The signal is resolved to its batch field throughsignal_fields(). DefaultNone(no teacher target; name a teacher field intarget_keysto compare against a teacher).supervised_loss (Callable[..., BaseLossFunction] | None, optional) – Factory, called as
supervised_loss(target_key=...), that builds the loss term driving the validation pass. DefaultNone(the quantity is a diagnostic that never enters the pass’s loss).
Examples
>>> from nvalchemi.training.distillation.evaluation import ( ... AccuracyQuantitySpec, ... ) >>> charges = AccuracyQuantitySpec( ... "charges", prediction_key="predicted_charges", reference_key="charges" ... ) >>> charges.supervised_loss is None True