Mtp State Scatter Kernels#
-
struct MtpLayerInfo#
Per-layer pointer bundle for batched MTP state scatter (analogous to KVLayerInfo).
Public Members
-
void *recurrentDst#
Main recurrent state pool [residentPoolRows, recElements] FP32.
-
void *recurrentSrc#
Recurrent replay stash, with layout owned by the recurrent backend.
-
void *convDst#
Main conv state pool [residentPoolRows, convElements] FP16.
-
void *convSrc#
Intermediate conv state buffer [batchSize, verifyTreeSize, convElements] FP16.
-
void *recurrentDst#
- void trt_edgellm::kernel::mtpScatterConvStates(
- MtpLayerInfo const *deviceLayerInfos,
- int32_t numLayers,
- int32_t activeBatchSize,
- int32_t residentPoolRows,
- int32_t verifyTreeSize,
- int32_t stateElements,
- int32_t const *acceptLengths,
- int32_t const *stateIndices,
- cudaStream_t stream
Commit each accepted conv prefix endpoint to its selected resident row. Zero-length prefixes preserve the resident state. stateElements must be divisible by 8.
- void trt_edgellm::kernel::mtpScatterAcceptedTreeConvStates(
- MtpLayerInfo const *deviceLayerInfos,
- int32_t numLayers,
- int32_t activeBatchSize,
- int32_t residentPoolRows,
- int32_t verifyTreeSize,
- int32_t stateElements,
- int32_t const *acceptedStateNodeIds,
- int32_t maxAcceptLen,
- int32_t const *acceptLengths,
- int32_t const *stateIndices,
- cudaStream_t stream
Commit the last non-negative node within each accepted path to its selected resident conv row. Paths without an accepted node preserve the resident state. stateElements must be divisible by 8.