Gdn Conv Commit Policy#
-
class GdnConvCommitPolicy : public trt_edgellm::rt::RecurrentCommitPolicy#
GDN commit: snapshot the per-token full recurrent state, then advance the persistent state along the accepted path by replaying the per-node stash the verify pass wrote at the head of each layer’s snapshot buffer. Convolution state commits by scatter.
Public Functions
- GdnConvCommitPolicy(
- Config config,
- std::vector<rt::Tensor> &liveRecurrent,
- std::vector<rt::Tensor> &liveConv,
- cudaStream_t stream
-
virtual bool hasIntermediateRecurrentStates() const noexcept override#
Whether verify-pass recurrent side buffers were allocated (spec-verify enabled).
-
virtual bool usesReplay() const noexcept override#
Whether the recurrent commit reconstructs by replay (mamba2) rather than scatter (gdn).
- virtual void commitLinear( ) override#
Commit the single accepted recurrent/conv state per sequence after linear (chain) verification.
- Parameters:
acceptLengths – [activeBatchSize] INT32 GPU accepted-token count per sequence.
stateIndices – [activeBatchSize] INT32 GPU active-row to resident-slot map.
- virtual void commitTree(
- rt::Tensor const &acceptedNodeIds,
- rt::Tensor const &acceptLengths,
- rt::Tensor const &stateIndices,
- int32_t verifySize,
- cudaStream_t stream
Commit the accepted recurrent/conv state per sequence after tree verification.
- Parameters:
acceptedNodeIds – [activeBatchSize, maxAcceptLen] INT32 GPU verify-node id per accepted token.
acceptLengths – [activeBatchSize] INT32 GPU accepted-token count per sequence.
stateIndices – [activeBatchSize] INT32 GPU active-row to resident-slot map.
verifySize – The verify-tree node count; selects snapshot vs chunk-replay for gdn.