Fp4 Quantize#

void trt_edgellm::kernel::fp4Quantize( rt::Tensor const&, rt::Tensor const&, rt::Tensor&, rt::Tensor&, cudaStream_t, )#

Quantize BF16/FP16 to packed FP4 with swizzled FP8 E4M3 scale factors. SM100+: hardware E2M1 conversion. Pre-SM100: software fallback. input shape [M, N]: M must be multiple of 128, N must be multiple of 16. input dataType must be kBF16 or kHALF.

globalSF is the forward-direction activation global scale (e.g. max|x|/(448*6)); the reciprocal consumed by the FP4 mapping is computed inside the kernel via a single IEEE divide per thread.

void trt_edgellm::kernel::fp4QuantizeLinearSF( rt::Tensor const&, rt::Tensor const&, rt::Tensor&, rt::Tensor&, cudaStream_t, )#: Quantize BF16/FP16 to packed FP4 with linear FP8 E4M3 scale factors. input shape [M, N]: N must be multiple of 16. Unlike fp4Quantize, outputSF is row-major [M, N / 16] because the SM110 TRT-LLM-style FC1 gather kernel loads A scale factors through a linear SFA layout.

void trt_edgellm::kernel::fp4QuantizeRoutedLinearSF( rt::Tensor const&, rt::Tensor const&, rt::Tensor const&, rt::Tensor&, rt::Tensor&, cudaStream_t, )#: Quantize FP16/BF16 token rows to routed packed FP4 rows with linear FP8 E4M3 scale factors. input is [T, N], topkIds is [T, topK], expertGlobalSF is [E], outputFP4 is [T * topK, N / 2], and outputSF is row-major [T * topK, N / 16]. Routed row token * topK + slot uses expertGlobalSF[topkIds[token, slot]].

void trt_edgellm::kernel::fp4BuildLayoutAndQuantizeRoutedLinearSFDecode( rt::Tensor const&, rt::Tensor const&, rt::Tensor const&, MoELayoutBuffers&, rt::Tensor&, rt::Tensor&, int32_t, int32_t, cudaStream_t, )#

Decode-specialized setup path for SM110 NVFP4 MoE.

Combines the compact MoE layout build and routed linear-SF FP4 quantization into one CUDA launch. V1 is intended for decode (input.shape[0] == 1) with small topK, where clearing the full padded layout dominates setup time.