Activations

Activations are non-linear functions that transform the input data into a new representation.

The following table lists the available activations:

CELU	Continuously differentiable Exponential Linear Unit (CELU) activation function.
ELU	Exponential Linear Unit (ELU) activation function.
GELU	Gaussian Error Linear Unit (GELU) activation function.
HardSigmoid	Hard Sigmoid activation function.
HardSwish	Hard Swish (HardSwish) activation function.
LeakyReLU	Leaky Rectified Linear Unit (Leaky ReLU) activation function.
LogSoftmax	Log-Softmax activation function.
Mish	Mish activation function.
PReLU	Parametric ReLU (PReLU) operation.
ReLU	Rectified Linear Unit (ReLU) activation function.
SELU	Scaled Exponential Linear Unit (SELU) activation function.
Shrink	Shrink activation function.
Sigmoid	Sigmoid activation function.
Softmax	Softmax activation function.
Softplus	Softplus activation function.
Softsign	Softsign activation function.
Swish	Swish activation function.
Threshold	Threshold activation function.