Dflash2 Candidate Selector#

void trt_edgellm::kernel::launchDFlash2CandidateSelector(
rt::Tensor const &candidateIds,
rt::Tensor const &unaryLogits,
rt::Tensor const &projectedHidden,
rt::Tensor const &anchorTokenIds,
rt::Tensor const &predecessorCodebook,
rt::Tensor const &successorCodebook,
rt::Tensor const &uniforms,
rt::Tensor const &temperatures,
rt::Tensor const &greedyMask,
rt::Tensor &proposalTokenIds,
rt::Tensor &proposalProbabilities,
cudaStream_t stream
)#
void trt_edgellm::kernel::dflash2SelectorTreeCandidates(
rt::Tensor const &supportIds,
rt::Tensor const &unaryValues,
rt::Tensor const &projectedHidden,
rt::Tensor const &anchorTokenIds,
rt::Tensor const &predecessorCodebook,
rt::Tensor const &successorCodebook,
rt::Tensor &candidateTokenIds,
rt::Tensor &candidateLogProbs,
rt::Tensor &candidateChildStates,
cudaStream_t stream
)#

DDTree Stage A for DFlash2: selector-scored child lists for every (proposal row, parent state).

Row d’s parent states are the support indices of row d - 1, so a parent’s state is the predecessor token supportIds[b, d - 1, state]; row 0 has the single state 0 whose predecessor is the anchor. A child c of state i scores s = unary + <P[pred] * z, Q[c]>, bitwise equal to launchDFlash2CandidateSelector’s score, and carries log(softmax(s)) over the row’s valid support at temperature 1. Out-of-vocabulary and repeated support ids score -inf. Each list keeps the best candidateTopK by (score desc, token asc); childStates holds the candidate’s support index, the child’s state in row d + 1. Unused lists and slots are -inf.

Inputs: supportIds [B, P, S] INT32, unaryValues [B, P, S] FP32, projectedHidden [B, P, R] FP16, anchorTokenIds [B] INT32, predecessor/successor codebooks [V, R] FP16. Outputs: candidateTokenIds / candidateLogProbs / candidateChildStates [B, P, S, K], K <= S.