Nvfp4 Moe Types

struct MoELayout

CPU-side layout result. Populated by the test-only CPU reference builder.

Public Members

rt::Tensor tileIdxToGroupIdx

rt::Tensor tileIdxToMnLimit

rt::Tensor permutedIdxToExpandedIdx

rt::Tensor numNonExitingTiles

int32_t numTiles = {}

int32_t mPadded = {}

std::vector<int32_t> tileIdxToGroupIdxHost

std::vector<int32_t> tileIdxToMnLimitHost

struct MoELayoutBuffers

Pre-allocated GPU buffers for the layout builder kernel. maxTiles = tileIdxToGroupIdx.getShape()[0], maxMPadded = permutedIdxToExpandedIdx.getShape()[0].

Public Members

rt::Tensor tileIdxToGroupIdx

rt::Tensor tileIdxToMnLimit

rt::Tensor permutedIdxToExpandedIdx

rt::Tensor numNonExitingTiles