Nvfp4 Moe Scale Transform#
- cudaError_t trt_edgellm::kernel::launchNvfp4MoeScaleTransformSourceBatch(
- uint8_t const *const *firstSources,
- uint8_t const *const *secondSources,
- int32_t count,
- int8_t *mmaLayout,
- int32_t rows,
- int32_t Ksf,
- Nvfp4MoeFc1Layout layout,
- cudaStream_t stream
Transform non-contiguous checkpoint scales into an NVFP4 MoE MMA layout.
secondSourcesis null for FC2; otherwiselayoutselects the FC1 arrangement required by the consuming plugin.
- cudaError_t trt_edgellm::kernel::launchNvfp4MoeScaleTransformSourceBatchPadded(
- uint8_t const *const *firstSources,
- uint8_t const *const *secondSources,
- int32_t count,
- int8_t *mmaLayout,
- int32_t sourceRows,
- int32_t sourceKsf,
- int32_t outputRows,
- int32_t outputKsf,
- Nvfp4MoeFc1Layout layout,
- cudaStream_t stream
Transform scales while zero-padding to larger plugin M/K dimensions.
- cudaError_t trt_edgellm::kernel::launchNvfp4MoeScaleNormalizeSourceBatchPadded(
- uint8_t const *const *firstWeights,
- uint8_t const *const *secondWeights,
- uint8_t const *const *firstScales,
- uint8_t const *const *secondScales,
- float const *firstAlphas,
- float const *secondAlphas,
- int32_t count,
- int8_t *mmaLayout,
- int32_t sourceRows,
- int32_t sourceRowBytes,
- int32_t sourceScaleColumns,
- int32_t outputRows,
- int32_t outputScaleColumns,
- Nvfp4MoeFc1Layout layout,
- cudaStream_t stream
Requantize provider-packed FP4 blocks and write their normalized FP8 scales directly in the final plugin layout.
secondWeightsis null for FC2.