medusa_model

Medusa model to support medusa decoding.

Classes

MedusaModel

Base Medusa Model.

class MedusaModel

Bases: DynamicModule

Base Medusa Model.

modify(medusa_num_heads=0, medusa_num_layers=0)

Base Medusa Model modify function. Child class should implement the details.