config

Configurations for speculative decoding modes.

ModeloptConfig MedusaConfig

Bases: ModeloptBaseConfig

Medusa config.

Show default config as JSON

Default config (JSON):

{
   "medusa_num_heads": 2,
   "medusa_num_layers": 1
}

field medusa_num_heads: int

Show details

The number of medusa heads added to the model.

field medusa_num_layers: int

Show details

The number of ResBlocks used in medusa head.