Spec Ple Staging#
-
class SpecPleStaging#
Host staging for the PLE preprocessors that read speculative verify rows on the host (Gemma4 host-token lookups, the Qwen4-Exp n-gram injection).
Public Functions
-
SpecPleStaging() = default#
- SpecPleStaging(
- DecodingRuntimeContext const &runtime,
- int32_t maxVerifyTokens,
- SpecVerifyLayout layout
Call once the base engine and its PipelineIO exist.
- void embed(
- DecodingRuntimeContext &runtime,
- DecodingInferenceContext const &context,
- Tensor const &tokenIds,
- int32_t requestCount,
- int32_t tokensPerRequest,
- cudaStream_t stream
Feed the verify rows to the PLE preprocessors; call after the ragged bindings, which fill the query lengths that mask padding.
-
SpecPleStaging() = default#