Reuse Plan#

struct HybridCheckpointCandidate#

Exact digest computed by the runtime for one stored hybrid candidate length.

Public Members

int32_t exactLength = {}#

BlockHash exactPrefixDigest = {}#

struct SpecReplayDependency#

Original coherent state boundary required by full-page replay.

Public Members

int32_t pathBlockCount = {}#

BlockHash terminalHash = {}#

struct ReusePlan#

Side-effect-free proposal for binding cached pages and allocating request-private state.

The free planning helpers expose deterministic metadata decisions for focused tests. ContextCacheManager builds and consumes each plan inside one serialized acquire call, so runtime callers never retain a mutable plan across metadata operations. The demand contains only resources that still need private allocation.

Public Members

ReusePlanMode mode = {ReusePlanMode::kVanilla}#

int32_t matchedTokenLength = {}#: Longest coherent cache match before any decoder-required rewind/replay adjustment.

int32_t reuseTokenLength = {}#: Materialized state boundary from which this request will actually execute.

std::vector<BlockHash> matchedBlockHashes#

std::vector<PageId> basePageBindings#

std::optional<RecordId> specRecord#

std::vector<PageId> specPageBindings#

std::optional<SpecReplayDependency> specReplayDependency#

bool hybridHasAttention = {false}#

std::optional<HybridCheckpointKey> hybridCheckpoint#

std::optional<RecordId> hybridRecord#

std::optional<int32_t> recurrentSnapshotBinding#

std::optional<int32_t> partialKvSnapshotBinding#

ResourceDemand demand#

ReusePlanKind kind = {ReusePlanKind::kStandard}#

SpecReplayMode specReplayMode = {SpecReplayMode::kNone}#

ReusePlan trt_edgellm::rt::makeVanillaReusePlan( std::vector<BlockHash> const &inputFullBlockHashes, int32_t inputTokenCount, int32_t pageSize, BaseBlockIndex const &index, ContextCacheLookupPolicy lookupPolicy )#

Build a base-model KV reuse plan for vanilla autoregressive decoding without mutating cache metadata.

An exact block-aligned full-input match rewinds one full page so the caller can recompute the final token boundary.

ReusePlan trt_edgellm::rt::makeHybridReusePlan( std::vector<HybridCheckpointCandidate> const &candidates, std::vector<BlockHash> const &inputFullBlockHashes, int32_t inputTokenCount, int32_t pageSize, bool hasAttention, CacheRecordStore const &records, ContextCacheLookupPolicy lookupPolicy )#: Build an exact hybrid/pure-recurrent reuse plan. Candidates may be unordered; the longest coherent checkpoint strictly shorter than the input wins. A missing snapshot member makes that candidate a complete miss.

ReusePlan trt_edgellm::rt::makeHybridMtpReusePlan( std::vector<HybridCheckpointCandidate> const &candidates, std::vector<BlockHash> const &inputFullBlockHashes, int32_t inputTokenCount, int32_t pageSize, CacheRecordStore const &records, ContextCacheLookupPolicy lookupPolicy = ContextCacheLookupPolicy::kUseCache )#: Build an exact hybrid+MTP reuse plan while retaining the successor-dependent boundary token privately.