external_model

External draft model to support speculative decoding with a standalone draft.

Classes

ExternalDraftModel

Base external draft model.

class ExternalDraftModel

Bases: DynamicModule

Base external draft model.

Unlike Eagle/Medusa/DFlash, the converted module is the draft: a pretrained causal LM that owns its embeddings and lm_head. The base (target) model is never grafted onto it; it contributes only cached hidden states, which are projected by the target lm_head to obtain teacher logits.

modify(config)

Base external draft modify function. Child class should implement the details.