Cute Dsl GEMM Nv Fp4 Runner#

class CuteDslGemmNvFp4Runner#

Public Functions

CuteDslGemmNvFp4Runner(int32_t smVersion, int32_t mmaTilerN = 64)#

Construct for one runtime SM and the desired MMA N-tile. The N-tile must be one of {64, 128}.

~CuteDslGemmNvFp4Runner()#
cudaError_t run(
void const *a,
void const *b,
void const *aSF,
void const *bSF,
void *d,
void const *alpha,
int32_t M,
int32_t N,
int32_t K,
cudaStream_t stream
)#

Launch the blockscaled NVFP4 GEMM with FP16 output.

Parameters:
  • a – FP4 packed activation, device ptr to [M, K/2] uint8

  • b – FP4 packed weight, device ptr to [N, K/2] uint8

  • aSF – UE4M3 atom-layout SF buffer for A, device ptr

  • bSF – UE4M3 atom-layout SF buffer for B, device ptr

  • d – FP16 output buffer, device ptr to [M, N] half

  • alpha – FP32 scalar applied in the SM110/SM12x epilogue before converting the FP32 accumulator to the output dtype.

  • M, N, K – dynamic shapes. N must be multiple of 128 and K a multiple of 64 (satisfied by TP layouts). M may be any positive integer.

  • stream – CUDA stream

cudaError_t runRawSfaM1(
void const *a,
void const *b,
void const *aSFRaw,
void const *bSF,
void *d,
void const *alpha,
int32_t M,
int32_t N,
int32_t K,
cudaStream_t stream
)#

Launch the SM12x decode specialization with a row-major activation scale. The kernel stages the single live SFA row directly into shared memory; M must be exactly 1. B scales remain in SfAtom layout.

cudaError_t runFp8(
void const *a,
void const *b,
void const *aSF,
void const *bSF,
void *d,
void const *alpha,
int32_t M,
int32_t N,
int32_t K,
cudaStream_t stream
)#

Launch the blockscaled NVFP4 GEMM with FP8 E4M3 output.

Same A / B / SFA / SFB layout as run(); only the output dtype differs. The kernel’s epilogue casts FP32 accumulator → FP8 E4M3.

Parameters:
  • d – FP8 E4M3 output, device ptr to [M, N] uint8 (1 byte/elem)

  • other – params same as run()

Public Static Functions

static bool loadKernelModules()#

Load all AOT kernel modules. Thread-safe, idempotent. Call once per process before the first run().

static void unloadKernelModules()#

Unload modules (call at process shutdown if resource cleanup matters).

static bool canImplement(int32_t smVersion, int32_t mmaTilerN)#

Is there an AOT variant compiled for this GPU + tile combo?

static bool canImplementRawSfaM1(int32_t smVersion)#

Is the SM12x M=1 row-major activation-scale specialization compiled?