Reuse Plan#

struct HybridCheckpointCandidate#

Exact digest computed by the runtime for one stored hybrid candidate length.

Public Members

int32_t exactLength = {}#

BlockHash exactPrefixDigest = {}#

struct SpecReplayDependency#

Original coherent draft boundary required by EAGLE full-page replay.

Full-page replay binds one fewer page than the original draft match, but the last retained draft slot still depends on the first token in this boundary. Publication checks this boundary hash before retaining the replayed path.

Public Members

BlockHash terminalHash = {}#

int32_t pathBlockCount = {}#

struct ReusePlan#

Side-effect-free proposal for binding cached pages and allocating request-private state.

The free planning helpers expose deterministic metadata decisions for focused tests. ContextCacheManager builds and consumes each plan inside one serialized acquire call, so runtime callers never retain a mutable plan across metadata operations. The demand contains only resources that still need private allocation.

Public Members

ReusePlanMode mode = {ReusePlanMode::kVanilla}#

int32_t matchedTokenLength = {}#: Longest coherent cache match before any decoder-required rewind/replay adjustment.

int32_t reuseTokenLength = {}#: Materialized state boundary from which this request will actually execute.

std::vector<BlockHash> matchedBlockHashes#

std::vector<PageId> basePageBindings#

std::optional<RecordId> draftRecord#

std::vector<PageId> draftPageBindings#

std::optional<SpecReplayDependency> specReplayDependency#

bool hybridHasAttention = {false}#

std::optional<HybridCheckpointKey> hybridCheckpoint#

std::optional<RecordId> hybridRecord#

std::optional<int32_t> recurrentSnapshotBinding#

std::optional<int32_t> partialKvSnapshotBinding#

ResourceDemand demand#

ReusePlanKind kind = {ReusePlanKind::kStandard}#

SpecReplayMode specReplayMode = {SpecReplayMode::kNone}#

ReusePlan trt_edgellm::rt::makeVanillaReusePlan( std::vector<BlockHash> const &inputFullBlockHashes, int32_t inputTokenCount, int32_t pageSize, BaseBlockIndex const &index, ContextCacheLookupPolicy lookupPolicy )#

Build a base-model KV reuse plan for vanilla autoregressive decoding without mutating cache metadata.

An exact block-aligned full-input match rewinds one full page so the caller can recompute the final token boundary.

ReusePlan trt_edgellm::rt::makeHybridReusePlan( std::vector<HybridCheckpointCandidate> const &candidates, std::vector<BlockHash> const &inputFullBlockHashes, int32_t inputTokenCount, int32_t pageSize, bool hasAttention, CacheRecordStore const &records, ContextCacheLookupPolicy lookupPolicy )#: Build an exact hybrid/pure-recurrent reuse plan. Candidates may be unordered; the longest coherent checkpoint strictly shorter than the input wins. A missing snapshot member makes that candidate a complete miss.

ReusePlan trt_edgellm::rt::makeSpecReusePlan( std::vector<BlockHash> const &inputFullBlockHashes, int32_t inputTokenCount, int32_t pageSize, BaseBlockIndex const &baseIndex, DraftPathIndex const &draftIndex, CacheRecordStore const &records, ContextCacheLookupPolicy lookupPolicy )#

Build a speculative reuse plan without mutating cache metadata.

Build plans for greedy, non-hybrid EAGLE. A hit requires one coherent draft record path; base-only state is intentionally ignored because it cannot reconstruct historical EAGLE draft KV.