conversion#

External draft conversion/restore utilities.

Functions

`convert_to_external_draft_model`	Convert a pretrained causal LM into a trainable external draft as per config.
`restore_external_draft_model`	Function for restoring a previously converted model to an external draft model.

convert_to_external_draft_model(model, config)#

Convert a pretrained causal LM into a trainable external draft as per config.

Note that, unlike the other speculative modes, model here is the draft rather than the base model. Nothing is grafted onto a target; the draft keeps its own embeddings and lm_head.

Parameters:

model (Module)
config (ExternalDraftConfig)

Return type:

tuple[Module, dict[str, Any]]

restore_external_draft_model(model, config, metadata)#

Function for restoring a previously converted model to an external draft model.

Parameters:

model (Module)
config (ExternalDraftConfig)
metadata (dict[str, Any])

Return type:

Module