Context Cache Config#

struct ContextCacheConfig#

Runtime configuration for one deployment-scoped context cache.

One enabled runtime is one trusted cache domain. Callers that require tenant isolation must use separate runtime instances; request-level cache-domain partitioning is not part of this interface.

Public Members

bool enabled = {false}#

Leave the existing identity-page runtime path unchanged when false.

int32_t maxRecords = {1024}#

Maximum retained publication endpoints; zero keeps no reusable records.

int32_t maxSnapshots = {}#

Total snapshots kept on a hybrid model; zero keeps four times the max batch size.

int64_t encoderEmbeddingCacheBudgetBytes = {256LL * 1024 * 1024}#

Device-memory budget for encoder embedding cache (ViT/audio). Zero disables.