bmm2_qdq

NVFP4 operand helpers for the attention P @ V matmul (BMM2).

P and V share the low-level nvfp4_scalar_qdq primitive, but retain thin operand-specific wrappers because their layouts and amax reductions differ. P is nonnegative with layout [M, K]; V is signed with layout [K, N]. Both use block-16 scaling along the BMM2 contraction axis.

Functions

fake_quant_v_onwrite

NVFP4-finalize complete block-16 groups in [v_lo, v_hi) in place.
fake_quant_v_onwrite(v_cache, block_table, v_lo, v_hi, *, max_new_tokens, page_size=16, v_qdq_scale=1.0)

NVFP4-finalize complete block-16 groups in [v_lo, v_hi) in place.

max_new_tokens is host metadata used to size the masked launch grid. The grid covers every group that the largest query chunk can complete without reading device metadata. v_lo and v_hi must describe aligned, completed block-16 boundaries; their device values are not host-validated.

Parameters:

  • v_cache (Tensor)

  • block_table (Tensor)

  • v_lo (Tensor)

  • v_hi (Tensor)

  • max_new_tokens (int)

  • page_size (int)

  • v_qdq_scale (float)

Return type:

None