Action KV Batch

class ActionKvBatchCollector

Collects request-local terminal KV rows and materializes the final dense action batch.

Public Functions

ActionKvBatchCollector(

int32_t maxBatch,

int32_t maxPagesPerSeq,

int32_t numPages

)

void beginRequest(

std::vector<bool> const &actionSlots,

std::vector<int64_t> const &mropeDeltas

)

void captureFinished(

KVPageTable const &pageTable,

Tensor const &deviceKvLengths,

std::vector<int8_t> const &finished,

std::vector<int32_t> const &originalIndices,

cudaStream_t stream

)

void completeCapture()

Consume queued length copies after the caller has synchronized the enqueue stream.

ActionKvBatchView materialize(cudaStream_t stream)

Validate completeness and materialize one dense, action-local logical KV batch.

std::vector<int32_t> const &originalRequestIndices(

) const noexcept

struct ActionKvBatchView

Dense logical KV batch consumed by an action model after language generation completes.

Public Members

KVPageTable const &pageTable

Tensor const &kvLengthsDevice

Tensor const &kvLengthsHost

std::vector<int64_t> const &mropeDeltas

int32_t batchSize = {}