gelu¶

nvtripy.gelu(input: Tensor) → Tensor[source]¶

Applies an approximated Gaussian Error Linear Units (GELU) function to each element of the input tensor:

\(\text{gelu}(x) = 0.5 * x * (1 + \tanh(\sqrt{2 / \pi} * (x + 0.044715 * x^3)))\)

Parameters:: input (Tensor) – [dtype=T1] The input tensor.
Returns:: [dtype=T1] A tensor of the same shape as the input.
Return type:: Tensor

TYPE CONSTRAINTS:

T1: float16, bfloat16, float32

Example

input = tp.Tensor([1.0, 2.0, 3.0, 4.0], dtype=tp.float32)
output = tp.gelu(input)

Local Variables¶

>>> input
tensor([1.0000, 2.0000, 3.0000, 4.0000], dtype=float32, loc=gpu:0, shape=(4,))

>>> output
tensor([0.8412, 1.9546, 2.9964, 3.9999], dtype=float32, loc=gpu:0, shape=(4,))