distillation

Modules

modelopt.torch.puzzletron.distillation.dataset

Real Puzzle-KD text dataset adapter for native AutoModel global KD.

modelopt.torch.puzzletron.distillation.flash_kld

Exact memory-bounded CE and KLD from hidden states and LM heads.

modelopt.torch.puzzletron.distillation.global_automodel

modelopt.torch.puzzletron.distillation.global_kd_recipe

Small Puzzletron extensions to NeMo AutoModel's native KD recipes.

modelopt.torch.puzzletron.distillation.loss

modelopt.torch.puzzletron.distillation.synthetic_vlm

Small deterministic image/text dataset for global-KD integration runs.

modelopt.torch.puzzletron.distillation.tournament

Exact-LM finalist selection and sequential native global KD.

Global distillation stage interfaces.