Nvfp4 A16 Blackwell GEMM Runner

class Nvfp4A16BlackwellGemmRunner

Dense W4A16 GEMM for the export-time SM110 opaque weight layout.

Logical tensors are activation [numTokens, inFeatures], weight [outFeatures, inFeatures], and output [numTokens, outFeatures]. The opaque inputs are qweight [N/128, K/64, 128, 32] packed E2M1 bytes and blockScales [N/128, K/64, 128, 4] raw E4M3 bytes. globalScale is a device FP32 scalar. AOT variants bake only the activation dtype and post-transpose MMA (TM,TN,TK) tile. Logical M=numTokens maps to MMA N. Runtime N/K define the tensor descriptors after the runner validates their positive, aligned dimensions.

Public Static Functions

static cudaError_t prepare(

Nvfp4A16BlackwellDtype dtype,

int32_t numTokens,

int32_t outFeatures,

int32_t inFeatures,

cudaStream_t stream

) noexcept

Load only the AOT module selected by this runtime dtype and token tile. Call this outside CUDA graph capture to prewarm the exact variant that run() will dispatch.

static cudaError_t loadKernelModules(cudaStream_t stream) noexcept

Legacy test helper that eagerly loads every compiled variant. Runtime plugin paths must use prepare() instead.

static bool isSupported(

int32_t smVersion,

Nvfp4A16BlackwellDtype dtype,

int32_t numTokens,

int32_t outFeatures,

int32_t inFeatures

) noexcept

static Nvfp4A16BlackwellTokenTile selectTokenTile(

int32_t numTokens

) noexcept

static size_t getWorkspaceSize(

Nvfp4A16BlackwellGemmParams const &params

) noexcept

static cudaError_t run(

Nvfp4A16BlackwellGemmParams const &params,

void *workspace,

size_t workspaceSize,

cudaStream_t stream

) noexcept

struct Nvfp4A16BlackwellGemmParams

Public Members

void const *activation

void const *qweight

void const *blockScales

float const *globalScale

void *output

int32_t numTokens

int32_t outFeatures

int32_t inFeatures

Nvfp4A16BlackwellDtype dtype