Compare
assert_model_equivalence(ckpt_path, model_tag, precision='fp32', rtol=None, atol=None)
Testing utility to compare the outputs of a NeMo2 checkpoint to the original HuggingFace model weights.
Compares the cosine similarity of the logit and hidden state outputs of a NeMo2 model checkpoint to the outputs of the corresponding HuggingFace model.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
ckpt_path
|
Path | str
|
A path to a NeMo2 checkpoint for an ESM-2 model. |
required |
model_tag
|
str
|
The HuggingFace model tag for the model to compare against. |
required |
precision
|
PrecisionTypes
|
The precision type to use for the comparison. Defaults to "fp32". |
'fp32'
|
rtol
|
float | None
|
The relative tolerance to use for the comparison. Defaults to None, which chooses the tolerance based on the precision. |
None
|
atol
|
float | None
|
The absolute tolerance to use for the comparison. Defaults to None, which chooses the tolerance based on the precision. |
None
|
Source code in bionemo/esm2/testing/compare.py
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 |
|