Spec Common State Tracker#
-
class SpecCommonStateTracker#
Tracks the greatest per-slot prefix materialized by both target and draft models.
Public Functions
-
void initialize(DecodingInferenceContext const &context)#
-
void reserve(int32_t rows)#
Allocate room for
rowsrows up front, so appendRow() and compact() within it never allocate.
-
void appendRow(int32_t row)#
A sequence joins a running batch at
row, its last row: track it from nothing until seatRow().
- void seatRow(
- int32_t row,
- int32_t materializedLength,
- int32_t pendingLength
The sequence at
rowseated its draft: both models hold its firstmaterializedLengthpositions, and its draft takes the nextpendingLengthat the next decode step, like a resident’s accepted span.
-
void materializePending(int32_t activeBatchSize)#
- void recordAccepted(
- int32_t const *hostAcceptLengths,
- int32_t activeBatchSize
- void compact(
- std::vector<int32_t> const &batchMapping,
- int32_t oldActiveBatch,
- int32_t newActiveBatch
-
void reset() noexcept#
-
void markDraftPrefillOutputsPending() noexcept#
-
void consumeDraftPrefillOutputs() noexcept#
-
bool draftPrefillOutputsPending() const noexcept#
- bool shouldUsePendingPrefillProposal(
- int32_t generationRound
- std::vector<int32_t> const &commonMaterializedStateLengths(
-
void initialize(DecodingInferenceContext const &context)#