medusa_model#

Medusa model to support medusa decoding.

Classes

MedusaModel

Base Medusa Model.
class MedusaModel#

Bases: DynamicModule

Base Medusa Model.

modify(medusa_num_heads=0, medusa_num_layers=0)#

Base Medusa Model modify function. Child class should implement the details.