Context Cache Config#
-
struct ContextCacheConfig#
Runtime configuration for one deployment-scoped context cache.
One enabled runtime is one trusted cache domain. Callers that require tenant isolation must use separate runtime instances; request-level cache-domain partitioning is not part of this interface.
Public Members
-
bool enabled = {false}#
Leave the existing identity-page runtime path unchanged when false.
-
int32_t maxRecords = {1024}#
Maximum retained publication endpoints; zero keeps no reusable records.
-
int32_t maxSnapshots = {}#
Total snapshots kept on a hybrid model; zero keeps four times the max batch size.
-
int64_t encoderEmbeddingCacheBudgetBytes = {256LL * 1024 * 1024}#
Device-memory budget for encoder embedding cache (ViT/audio). Zero disables.
-
bool enabled = {false}#