Runtime Module

API documentation for the runtime module.

• Audio Utils
• EAGLE Draft Engine Runner
• Hybrid Cache Manager
• Image Utils
• KV Cache Manager
• LLM Engine Runner
• LLM Inference Runtime
• LLM Inference Spec Decode Runtime
• LLM Runtime Utils
• Mamba Cache Manager
• Qwen3 Omni Tts Runtime
• Streaming