functional

Some supportive functions.

Classes

ClipFunction

An universal tensor clip function.

class ClipFunction

Bases: Function

An universal tensor clip function.

Pytorch’s clamp() only supports scalar range and doesn’t support broadcast. This implementation uses min/max which is more genaral. The gradient is defined according to IBM’s PACT paper https://arxiv.org/abs/1805.06085, which is also the behavior of Tensorflow’s clip_by_value()

static backward(ctx, grad_output): Backward pass for the clip function.

static forward(ctx, input, clip_value_min, clip_value_max): Forward pass for the clip function.