linear_attention

Modules

Linear-attention kernels for quantization.

fla_chunk_delta_h.py and fla_chunk_gated_delta_rule.py are adapted copies of the chunked GatedDeltaNet kernels of flash-linear-attention (fla.ops.common.chunk_delta_h and fla.ops.gated_delta_rule.chunk) that can fake-quantize the recurrent state carried between chunks to FP8 (state_qdq). They still import the surrounding fla operators, so fla-core==0.5.1 and Triton must be installed to use them. This package initializer does not import the kernels, so importing it needs neither.