KVCacheEntry

class sdm.cache.KVCacheEntry(key: Tensor, value: Tensor)

Cached key/value projections for a single transformer block.

Parameters:

• key (Tensor) – Cached key projection tensor.

• value (Tensor) – Cached value projection tensor.