Context Cache Config

struct ContextCacheConfig

Runtime configuration for one deployment-scoped context cache.

One enabled runtime is one trusted cache domain. Callers that require tenant isolation must use separate runtime instances; request-level cache-domain partitioning is not part of this interface.

Public Members

bool enabled = {false}

Leave the existing identity-page runtime path unchanged when false.

int32_t maxRecords = {1024}

Maximum retained publication endpoints; zero keeps no reusable records.

int64_t recurrentSnapshotPoolBytes = {}

Device-memory budget for preallocated recurrent/conv checkpoints.

int64_t partialKvSnapshotPoolBytes = {}

Device-memory budget for preallocated partial attention-KV checkpoints.