Iq2 Xxs Repack

cudaError_t trt_edgellm::kernel::launchIq2XxsMoeRepackSourceBatch(

uint8_t const *const *sources,

int32_t count,

int32_t *words,

uint16_t *bases,

int32_t N,

int32_t kBlocks,

cudaStream_t stream

)

Reorder a dense IQ2_XXS checkpoint matrix uint8 [N, K/256, 66] for W2A16GemmPlugin. values receives the ! payload as [N/P, K/256, P, 64] with P = 128 when N % 128 == 0, else 8; bases

receives the FP16 bases as ! [K/256, N]. Either output may be null. Bytes are only moved. cudaError_t launchIq2XxsGemmRepack(

uint8_t const* source, int8_t* values, uint16_t* bases, int32_t N, int32_t kBlocks, cudaStream_t stream);

! Reorder per-expert IQ2_XXS checkpoint matrices uint8 [N, K/256, 66] for W2A16MoePlugin. words receives each ! expert’s payload as int32 [N/64, K/128, 8, 4, 8, 2]; bases receives each expert’s FP16 bases as [K/256, N/16, 8, 2]. Either output may be null; both point at the first expert of this batch.