Cute Dsl Dual GEMM Runner

class CuteDslDualGemmRunner

Public Static Functions

static bool canImplement(int32_t smVersion) noexcept

True when the dual GEMM variants were compiled into this binary and the GPU is a supported Blackwell.

static cudaError_t prepare(

int32_t numTokens,

cudaStream_t stream

) noexcept

Load the AOT module that serves numTokens rows (idempotent, thread-safe). Must run outside CUDA graph capture; call it from configurePlugin / onShapeChange so enqueue only launches.

static cudaError_t run(

DualGemmBlackwellParams const &params,

cudaStream_t stream

) noexcept

Launch the variant selected by params.numTokens.

struct DualGemmBlackwellParams

Public Members

void const *activation

A [M, K] FP16, K contiguous.

void const *packedWeight

packed gate/up weight [2N, K] FP16 (dual_gemm_blackwell layout)

void *output

H [M, N] FP16, N contiguous.

int32_t numTokens

M.

int32_t interSize

N (intermediate size)

int32_t hiddenSize

K (hidden size)