Shrink#

API#

class warp_nn.modules.activations.Shrink(
*,
lambd: float = 0.5,
bias: float = 0.0,
requires_grad: bool = True,
)[source]#

Bases: Module

Shrink activation function.

This class computes the element-wise Shrink activation function:

\[\begin{split}\text{Shrink}(x) = \begin{cases} x + \text{bias}, & \text{ if } x < -\lambda\\ x - \text{bias}, & \text{ if } x > \lambda\\ 0, & \text{ otherwise} \end{cases}\end{split}\]

where

\[\lambda = \text{lambd}\]

PyTorch’s Hardshrink(lambd) and Softshrink(lambd) are equivalent to Shrink(lambd=lambd, bias=0) and Shrink(lambd=lambd, bias=lambd), respectively.

Parameters:

  • lambd – The lambda value (threshold) for the Shrink function.

  • bias – The bias value subtracted from (added to) the values above (below) the threshold.

  • requires_grad – Whether the cached output arrays of the module require gradients.

__call__(
input: array,
) → array[source]#

Forward pass of the activation function.

Parameters:

input – The input array, with up to 3 dimensions.

Returns:

The output array, with same shape as the input array.

Raises:

TypeError – If the input array’s data type or number of dimensions is not supported.

property bias: float[source]#

The bias value subtracted from (added to) the values above (below) the threshold.

property lambd: float[source]#

The lambda value (threshold) for the Shrink function.