layerwise_distillation_model
Meta-model wrapper to support layerwise-enabled knowledge-distillation learning.
Classes
Meta-model wrapper to support layerwise-enabled knowledge-distillation learning. |
- class LayerwiseDistillationModel
Bases:
DistillationModelMeta-model wrapper to support layerwise-enabled knowledge-distillation learning.
The LayerwiseDistillationModel is a subclass of the DistillationModel that injects teacher inputs into the corresponding student layers. This accomodates the case where the student model is the teacher with specific submodules replaced, which now need to be trained to mimic the original submodule in the teacher.
- export()
Export the distillation model.
- modify(*args, **kwargs)
Modify the distillation model.