Scaled Exponential Linear Unit (SELU)#

API#

class warp_nn.modules.activations.SELU(
*,
scale: float = 1.0507009873554805,
alpha: float = 1.6732632423543772,
requires_grad: bool = True,
)[source]#

Bases: Module

Scaled Exponential Linear Unit (SELU) activation function.

This class computes the element-wise SELU activation function:

\[\begin{split}\text{SELU}(x) = \begin{cases} \lambda \, x, & \text{ if } x \geq 0\\ \lambda \, \alpha \, (e^x - 1), & \text{ if } x < 0 \end{cases}\end{split}\]

where \(\lambda\) and \(\alpha\) default to \(1.0507009873554804934193349852946\) and \(1.6732632423543772848170429916717\) respectively.

Parameters:

  • scale – The scale (\(\lambda\)) value for the SELU function.

  • alpha – The alpha (\(\alpha\)) value for the SELU function.

  • requires_grad – Whether the cached output arrays of the module require gradients.

__call__(input: array) → array[source]#

Forward pass of the activation function.

Parameters:

input – The input array, with up to 3 dimensions.

Returns:

The output array, with same shape as the input array.

Raises:

TypeError – If the input array’s data type or number of dimensions is not supported.

property alpha[source]#

The alpha value for the SELU function.

property scale[source]#

The scale value for the SELU function.