Batch Evict Kernels

struct KVLayerInfo

Per-layer KV cache metadata for batched kernel operations.

Public Members

void *data

Pointer to this layer’s KV buffer [maxB, 2, H, S, D].

int32_t numKVHeads

Number of KV heads for this layer.

int32_t maxSeqLen

Max sequence length for this layer.