speculative

Modules

modelopt.torch.speculative.config

Configurations for speculative decoding modes.

modelopt.torch.speculative.eagle

Eagle Optimization Method.

modelopt.torch.speculative.medusa

Medusa Optimization Method.

modelopt.torch.speculative.mode

This module contains the mode descriptor for the quantization mode.

modelopt.torch.speculative.plugins

Handles speculative plugins for third-party modules.

modelopt.torch.speculative.redrafter

Redrafter Optimization Method.

modelopt.torch.speculative.speculative_decoding

User-facing API for converting a model into a modelopt.torch.speculative.MedusaModel.

modelopt.torch.speculative.utils

Utils for speculative decoding.

Speculative Decoding Optimizations.