extensions

Module to load C++ / CUDA extensions.

Functions

get_cuda_ext	Returns the cuda extension for tensor_quant.
get_cuda_ext_fp8	Returns the cuda extension for tensor_quant_fp8.
get_cuda_ext_ggml	Return the GGML-compatible IQ packing extension, exposing one packer per IQ format.
get_cuda_ext_mx	Returns the cuda extension for tensor_quant_mx.
precompile	Precompile the CUDA extensions.

get_cuda_ext(raise_if_failed=False)

Returns the cuda extension for tensor_quant.

Parameters:

raise_if_failed (bool)

get_cuda_ext_fp8(raise_if_failed=False)

Returns the cuda extension for tensor_quant_fp8.

Parameters:

raise_if_failed (bool)

get_cuda_ext_ggml(raise_if_failed=False)

Return the GGML-compatible IQ packing extension, exposing one packer per IQ format.

The formats share their packing helpers, CUDA version requirement, and build flags, so they build as a single extension: iq1_s_pack(input, grid) and iq2_xs_pack(input, grid, scales).

Parameters:

raise_if_failed (bool)

get_cuda_ext_mx(raise_if_failed=False)

Returns the cuda extension for tensor_quant_mx.

Parameters:

raise_if_failed (bool)

precompile()

Precompile the CUDA extensions.