functional
Some supportive functions.
Classes
An universal tensor clip function. |
|
The fast Hadamard transform. |
Functions
Normalized fast hadamard transform. |
- class ClipFunction
Bases:
Function
An universal tensor clip function.
Pytorch’s clamp() only supports scalar range and doesn’t support broadcast. This implementation uses min/max which is more genaral. The gradient is defined according to IBM’s PACT paper https://arxiv.org/abs/1805.06085, which is also the behavior of Tensorflow’s clip_by_value()
- static backward(ctx, grad_output)
Backward pass for the clip function.
- static forward(ctx, input, clip_value_min, clip_value_max)
Forward pass for the clip function.
- class FastHadamardTransform
Bases:
Function
The fast Hadamard transform.
This only works for inputs.shape[-1] == power of 2.
- static backward(ctx, grad_outputs)
Hadamard backward.
- static forward(ctx, inputs)
Hadamard forward.
- normalized_hadamard_transform(inputs)
Normalized fast hadamard transform.