nvfp4_act_headroom

Headroom calibrator for the NVFP4 activation global scale.

Classes

NVFP4ActHeadroomCalibrator

Calibrates the NVFP4 activation global scale with headroom for unseen outliers.
class NVFP4ActHeadroomCalibrator

Bases: _Calibrator

Calibrates the NVFP4 activation global scale with headroom for unseen outliers.

NVFP4 scales a tensor in two levels: an FP8-E4M3 scale per 16-element block, and one per-tensor global scale (amax). Plain max calibration sets the global scale from the largest block seen during calibration, which leaves no room above it: any activation larger than the calibration max saturates.

This calibrator instead anchors the global scale to a low percentile of the per-block amax distribution:

amax = max(rho * anchor, upper)

where anchor is the per-block amax at anchor_percentile and upper is the per-block amax at upper_percentile. Multiplying the low anchor by rho places the calibrated blocks in the lower part of the FP8 scale range and leaves the rest as upward headroom; upper is the top of the range the scale commits to representing.

upper_percentile defaults to 99.99 rather than the literal maximum on purpose. A single freak block far above the rest would otherwise drag the global scale up so far that every other block’s FP8 block scale falls below subnormal and flushes to zero – losing the whole tensor to protect one value. The default clips those rare blocks instead. Set upper_percentile=100 to use the literal observed max, which guarantees no calibration data is clipped at the cost of that exposure.

Parameters:

  • num_bits – quantizer num_bits ((2, 1) for NVFP4); kept for interface parity.

  • axis – unused (the global scale is per-tensor); kept for interface parity.

  • unsigned – unused; kept for interface parity.

  • block_size – NVFP4 block width along the last dim.

  • anchor_percentile – percentile of the per-block amaxes used as the anchor; must be > 0.

  • upper_percentile – percentile of the per-block amaxes used as the top of the calibrated range. 100 uses the literal observed max (no calibration data is clipped).

  • rho – headroom factor applied to the anchor. Must be in (0, 28672).

  • num_bins – number of log2 histogram bins.

  • log2_min – low end of the log2 range covered by the histogram.

  • log2_max – high end of the log2 range covered by the histogram.

__init__(num_bits=(2, 1), axis=None, unsigned=False, *, block_size=16, anchor_percentile=1.0, upper_percentile=99.99, rho=16384.0, num_bins=512, log2_min=-40.0, log2_max=40.0)

Initialize.

collect(x)

Accumulate the per-block amax histogram for one activation batch.

Parameters:

x (Tensor)

Return type:

None

compute_amax()

Return the calibrated NVFP4 activation global scale.

Return type:

Tensor | None

reset()

Reset the collected histogram.

Return type:

None