speculative

Modules

modelopt.torch.speculative.config

Configurations for speculative decoding modes.

modelopt.torch.speculative.eagle

Eagle Optimization Method.

modelopt.torch.speculative.medusa

Medusa Optimization Method.

modelopt.torch.speculative.mode

This module contains the mode descriptor for the quantization mode.

modelopt.torch.speculative.mtp

Eagle Optimization Method.

modelopt.torch.speculative.plugins

Handles speculative plugins for third-party modules.

modelopt.torch.speculative.speculative_decoding

User-facing API for converting a model into a modelopt.torch.speculative.MedusaModel.

modelopt.torch.speculative.utils

Utils for speculative decoding.

Speculative Decoding Optimizations.