Cute Dsl GEMM Nv Fp4 Runner#

class CuteDslGemmNvFp4Runner#

Public Functions

explicit CuteDslGemmNvFp4Runner(int32_t mmaTilerN = 64)#

Construct with the desired MMA N-tile. Must be one of {64, 128}.

~CuteDslGemmNvFp4Runner()#
cudaError_t run(
void const *a,
void const *b,
void const *aSF,
void const *bSF,
void *d,
int32_t M,
int32_t N,
int32_t K,
cudaStream_t stream,
)#

Launch the blockscaled NVFP4 GEMM with FP16 output.

Parameters:

  • a – FP4 packed activation, device ptr to [M, K/2] uint8

  • b – FP4 packed weight, device ptr to [N, K/2] uint8

  • aSF – UE4M3 atom-layout SF buffer for A, device ptr

  • bSF – UE4M3 atom-layout SF buffer for B, device ptr

  • d – FP16 output buffer, device ptr to [M, N] half

  • M, N, K – dynamic shapes. N must be multiple of 128 and K a multiple of 64 (satisfied by TP layouts). M may be any positive integer.

  • stream – CUDA stream

cudaError_t runFp8(
void const *a,
void const *b,
void const *aSF,
void const *bSF,
void *d,
int32_t M,
int32_t N,
int32_t K,
cudaStream_t stream,
)#

Launch the blockscaled NVFP4 GEMM with FP8 E4M3 output.

Same A / B / SFA / SFB layout as run(); only the output dtype differs. The kernel’s epilogue casts FP32 accumulator → FP8 E4M3.

Parameters:

  • d – FP8 E4M3 output, device ptr to [M, N] uint8 (1 byte/elem)

  • other – params same as run()

Public Static Functions

static bool loadKernelModules()#

Load all AOT kernel modules. Thread-safe, idempotent. Call once per process before the first run().

static void unloadKernelModules()#

Unload modules (call at process shutdown if resource cleanup matters).

static bool canImplement(int32_t smVersion, int32_t mmaTilerN)#

Is there an AOT variant compiled for this GPU + tile combo?