pruning
Modules
Model-agnostic surgery primitives for FFN-channel and attention-head pruning. |
|
Reversible compact forwards for physically sliced attention and GDN candidates. |
|
Generic (AnyModel) dynamic single-block pruning — no realized weights. |
|
Prefix-size sampling for nested (Matryoshka) elastic bypass distillation. |
|
Descriptor-owned residual hidden-width ranking, sorting, and slicing. |
|
|
|
|
|
Qwen GatedDeltaNet geometry and coupled permutation helpers. |
|
|
|
Weight-only surgery for latent-projected mixture-of-experts layers. |
|
Generic packed-tensor surgery for Mamba2-style mixers. |
|
Materialize a pruned variant from the sorted teacher — slice/merge, no realized checkpoints. |
|
Global per-layer-embedding (PLE) channel ranking and tensor surgery. |
|
Shared reversible application of one complete typed block candidate. |
|
Differentiable full-envelope execution for a nested residual width. |
|
Static-envelope runtime slicing for descriptor-owned PLE channels. |
|
Build a "sorted teacher": the teacher with each layer's FFN channels and attention heads reordered by importance (from activation scoring). |
Structured pruning mixins and checkpoint utilities for Puzzletron.