Spec Common State Tracker#

class SpecCommonStateTracker#

Tracks the greatest per-slot prefix materialized by both target and draft models.

Public Functions

void initialize(DecodingInferenceContext const &context)#
void reserve(int32_t rows)#

Allocate room for rows rows up front, so appendRow() and compact() within it never allocate.

void appendRow(int32_t row)#

A sequence joins a running batch at row, its last row: track it from nothing until seatRow().

void seatRow(
int32_t row,
int32_t materializedLength,
int32_t pendingLength
)#

The sequence at row seated its draft: both models hold its first materializedLength positions, and its draft takes the next pendingLength at the next decode step, like a resident’s accepted span.

void materializePending(int32_t activeBatchSize)#
void recordAccepted(
int32_t const *hostAcceptLengths,
int32_t activeBatchSize
)#
void compact(
std::vector<int32_t> const &batchMapping,
int32_t oldActiveBatch,
int32_t newActiveBatch
)#
void reset() noexcept#
void markDraftPrefillOutputsPending() noexcept#
void consumeDraftPrefillOutputs() noexcept#
bool draftPrefillOutputsPending() const noexcept#
bool shouldUsePendingPrefillProposal(
int32_t generationRound
) const noexcept#
std::vector<int32_t> const &commonMaterializedStateLengths(
) const noexcept#