Swa KV Cache State

class SwaWindowStateView

Non-owning access to the bounded manager’s settled per-sequence windows. The manager retains ownership and the view is valid only for the synchronous completion call in which it is supplied.

Public Functions

SwaWindowStateView() = default

inline SwaWindowStateView(

SwaKVCacheState const *states,

size_t size

) noexcept

inline size_t size() const noexcept

inline bool empty() const noexcept

inline SwaWindowState const &operator[](size_t slot) const

struct SwaCacheConfig

Optional engine-wide sliding-window context-cache configuration.

Public Members

int32_t windowSizeTokens = {}

int32_t replacementPageCount = {kSWA_REPLACEMENT_PAGES}

struct SwaPageBinding

One absolute logical-to-physical binding in a bounded SWA window.

Public Members

int32_t logicalPage = {}

PageId page = {}

struct SwaWindowState

Settled SWA state that can be stored immutably in a context-cache record.

Subclassed by trt_edgellm::rt::SwaKVCacheState

Public Members

int32_t pageSizeTokens = {}

int32_t windowSizeTokens = {}

int32_t exactResidentTokenCount = {}

int32_t retainedLogicalPageBegin = {}

int32_t retainedLogicalPageEnd = {}

std::deque<SwaPageBinding> pageBindings

SwaRetentionMode retentionMode = {SwaRetentionMode::kLive}

struct SwaKVCacheState : public trt_edgellm::rt::SwaWindowState

Request-local execution state. The inherited settled fields remain directly accessible to existing runtime code; the additional members are transient and are never stored in a cache record.

Public Members

ResidentRef resident

std::vector<PageId> reservedPages

std::vector<SwaPageBinding> newPageBindings

std::vector<SwaPageBinding> pendingRetireBindings

std::optional<int32_t> pendingEndpoint