Rectified Linear Unit (ReLU)

API

class warp_nn.modules.activations.ReLU(*, requires_grad: bool = True)

Bases: Module

Rectified Linear Unit (ReLU) activation function.

This class computes the element-wise ReLU activation function:

\[\begin{split}\text{ReLU}(x) = \begin{cases} x, & \text{ if } x \geq 0\\ 0, & \text{ if } x < 0 \end{cases}\end{split}\]

Parameters:

requires_grad – Whether the cached output arrays of the module require gradients.

__call__(input: array) → array

Forward pass of the activation function.

Parameters:

input – The input array, with up to 3 dimensions.

Returns:

The output array, with same shape as the input array.

Raises:

TypeError – If the input array’s data type or number of dimensions is not supported.