Swish#

API#

class warp_nn.modules.activations.Swish(*, alpha: float = 1.0, requires_grad: bool = True)[source]#

Bases: Module

Swish activation function.

This class computes the element-wise Swish activation function:

\[\text{Swish}(x) = x \, \sigma(\alpha \, x) = \frac{x}{1 + e^{-\alpha \, x}}\]

With the default alpha value, it is the Sigmoid Linear Unit (SiLU) activation function (PyTorch’s SiLU).

Parameters:

  • alpha – The coefficient that multiplies the input of the sigmoid function.

  • requires_grad – Whether the cached output arrays of the module require gradients.

__call__(
input: array,
) → array[source]#

Forward pass of the activation function.

Parameters:

input – The input array, with up to 3 dimensions.

Returns:

The output array, with same shape as the input array.

Raises:

TypeError – If the input array’s data type or number of dimensions is not supported.

property alpha: float[source]#

The coefficient that multiplies the input of the sigmoid function.