Is this page helpful?

activation.h#

Activation functions.

Enums

enum class NVTE_Activation_Type#

Computes activation of the input. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Param input:

[in] Input tensor for activation.

Param output:

[inout] Output tensor.

Param stream:

[in] CUDA stream used for the operation.

Values:

enumerator GELU#
enumerator GEGLU#
enumerator GLU#
enumerator SILU#
enumerator SWIGLU#
enumerator RELU#
enumerator REGLU#
enumerator QGELU#
enumerator QGEGLU#
enumerator SRELU#
enumerator SREGLU#
enumerator CLAMPED_SWIGLU#

Functions

void nvte_gelu(
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the GeLU activation of the input. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • input – [in] Input tensor for activation.

  • output – [inout] Output tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_group_gelu(
const NVTEGroupedTensor input,
NVTEGroupedTensor output,
cudaStream_t stream,
)#

Computes the GeLU activation of the grouped input. If the scaling mode of the grouped output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used. For grouped tensors with a varying last dimension, the last dimension must be a multiple of 128.

Parameters:

  • input – [in] Input grouped tensor for activation.

  • output – [inout] Output grouped tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_silu(
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the SiLU activation of the input. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • input – [in] Input tensor for activation.

  • output – [inout] Output tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_group_silu(
const NVTEGroupedTensor input,
NVTEGroupedTensor output,
cudaStream_t stream,
)#

Computes the SiLU activation of the grouped input. If the scaling mode of the grouped output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used. For grouped tensors with a varying last dimension, the last dimension must be a multiple of 128.

Parameters:

  • input – [in] Input grouped tensor for activation.

  • output – [inout] Output grouped tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_group_scaled_swiglu(
const NVTEGroupedTensor input,
const NVTETensor prob,
NVTEGroupedTensor output,
cudaStream_t stream,
)#

Grouped scaled SwiGLU “recompute” fused with MXFP8 columnwise quantization.

Computes, per token n and feature h: output[n, h] = ( silu(input[n, h]) * input[n, H + h] ) * prob[n] where the grouped input has logical shape [N, 2H] (last dim = [act | gate]) and the grouped output has logical shape [N, H]. Only the columnwise MXFP8 output is produced (it feeds the MoE FC2 weight-gradient GEMM). Restrictions: NVTE_MXFP8_1D_SCALING output, uniform H across experts (SAME_BOTH_DIMS / VARYING_FIRST_DIM), per-expert token counts divisible by 128. Scales may be compact or in the cuBLAS GEMM-swizzled layout; the swizzled layout additionally requires H divisible by 128 and, for multiple experts, VARYING_FIRST_DIM.

Parameters:

  • input – [in] Grouped input tensor [N, 2H] ([act|gate]).

  • prob – [in] Per-token weights, at least N elements, in the same dtype as input.

  • output – [inout] Grouped output tensor [N, H] (columnwise MXFP8).

  • stream – [in] CUDA stream used for the operation.

void nvte_group_scaled_clamped_swiglu(
const NVTEGroupedTensor input,
const NVTETensor prob,
NVTEGroupedTensor output,
float limit,
float alpha,
float glu_linear_offset,
cudaStream_t stream,
)#

Clamped variant of nvte_group_scaled_swiglu.

Computes, per token n and feature h, with x = input[n, h] and g = input[n, H + h]: a = min(x, limit) output[n, h] = ( a * sigmoid(alpha * a)

  • (clamp(g, -limit, limit) + glu_linear_offset) ) * prob[n] Note the asymmetry, which matches nvte_clamped_swiglu and the forward path of the gated MXFP8 kernels: the activation half is clamped from above only, the gate half on both sides and then offset. Same shape, layout and scaling-mode restrictions as nvte_group_scaled_swiglu.

Parameters:

  • input – [in] Grouped input tensor [N, 2H] ([act|gate]).

  • prob – [in] Per-token weights, at least N elements, in the same dtype as input.

  • output – [inout] Grouped output tensor [N, H] (columnwise MXFP8).

  • limit – [in] Clamp threshold, must be positive.

  • alpha – [in] Sigmoid gain of the activation half.

  • glu_linear_offset – [in] Offset added to the gate half after clamping.

  • stream – [in] CUDA stream used for the operation.

void nvte_relu(
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the ReLU activation of the input. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • input – [in] Input tensor for activation.

  • output – [inout] Output tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_group_relu(
const NVTEGroupedTensor input,
NVTEGroupedTensor output,
cudaStream_t stream,
)#

Computes the ReLU activation of the grouped input. If the scaling mode of the grouped output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used. For grouped tensors with a varying last dimension, the last dimension must be a multiple of 128.

Parameters:

  • input – [in] Input grouped tensor for activation.

  • output – [inout] Output grouped tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_qgelu(
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the Quick GeLU activation of the input. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • input – [in] Input tensor for activation.

  • output – [inout] Output tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_group_qgelu(
const NVTEGroupedTensor input,
NVTEGroupedTensor output,
cudaStream_t stream,
)#

Computes the Quick GeLU activation of the grouped input. If the scaling mode of the grouped output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used. For grouped tensors with a varying last dimension, the last dimension must be a multiple of 128.

Parameters:

  • input – [in] Input grouped tensor for activation.

  • output – [inout] Output grouped tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_srelu(
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the Squared ReLU activation of the input. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • input – [in] Input tensor for activation.

  • output – [inout] Output tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_group_srelu(
const NVTEGroupedTensor input,
NVTEGroupedTensor output,
cudaStream_t stream,
)#

Computes the Squared ReLU activation of the grouped input. If the scaling mode of the grouped output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used. For grouped tensors with a varying last dimension, the last dimension must be a multiple of 128.

Parameters:

  • input – [in] Input grouped tensor for activation.

  • output – [inout] Output grouped tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_dgelu(
const NVTETensor grad,
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the GeLU activation gradient. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • grad – [in] Incoming gradient.

  • input – [in] Input tensor for activation.

  • output – [inout] Output tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_group_dgelu(
const NVTEGroupedTensor grad,
const NVTEGroupedTensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the GeLU activation gradient of the grouped input. If the scaling mode of the output grouped tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used. For grouped tensors with a varying last dimension, the last dimension must be a multiple of 128.

Parameters:

  • grad – [in] Incoming grouped gradient.

  • input – [in] Input grouped tensor for activation.

  • output – [inout] Output grouped tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_dsilu(
const NVTETensor grad,
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the SiLU activation gradient. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • grad – [in] Incoming gradient.

  • input – [in] Input tensor for activation.

  • output – [inout] Output tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_group_dsilu(
const NVTEGroupedTensor grad,
const NVTEGroupedTensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the SiLU activation gradient of the grouped input. If the scaling mode of the output grouped tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used. For grouped tensors with a varying last dimension, the last dimension must be a multiple of 128.

Parameters:

  • grad – [in] Incoming grouped gradient.

  • input – [in] Input grouped tensor for activation.

  • output – [inout] Output grouped tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_drelu(
const NVTETensor grad,
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the ReLU activation gradient. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • grad – [in] Incoming gradient.

  • input – [in] Input tensor for activation.

  • output – [inout] Output tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_group_drelu(
const NVTEGroupedTensor grad,
const NVTEGroupedTensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the ReLU activation gradient of the grouped input. If the scaling mode of the output grouped tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used. For grouped tensors with a varying last dimension, the last dimension must be a multiple of 128.

Parameters:

  • grad – [in] Incoming grouped gradient.

  • input – [in] Input grouped tensor for activation.

  • output – [inout] Output grouped tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_dqgelu(
const NVTETensor grad,
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the Quick GeLU activation gradient. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • grad – [in] Incoming gradient.

  • input – [in] Input tensor for activation.

  • output – [inout] Output tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_group_dqgelu(
const NVTEGroupedTensor grad,
const NVTEGroupedTensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the Quick GeLU activation gradient of the grouped input. If the scaling mode of the output grouped tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used. For grouped tensors with a varying last dimension, the last dimension must be a multiple of 128.

Parameters:

  • grad – [in] Incoming grouped gradient.

  • input – [in] Input grouped tensor for activation.

  • output – [inout] Output grouped tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_dsrelu(
const NVTETensor grad,
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the Squared ReLU activation gradient. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • grad – [in] Incoming gradient.

  • input – [in] Input tensor for activation.

  • output – [inout] Output tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_group_dsrelu(
const NVTEGroupedTensor grad,
const NVTEGroupedTensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the Squared ReLU activation gradient of the grouped input. If the scaling mode of the output grouped tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used. For grouped tensors with a varying last dimension, the last dimension must be a multiple of 128.

Parameters:

  • grad – [in] Incoming grouped gradient.

  • input – [in] Input grouped tensor for activation.

  • output – [inout] Output grouped tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_glu(
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the GLU (Gated Linear Unit) activation of the input. GLU(a,b) = sigmoid(a) * b See “Language Modeling with Gated Convolutional Networks” (arXiv:1612.08083) and “GLU Variants Improve Transformer” (arXiv:2002.05202). If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • input – [in] Input tensor of shape [N, H * 2].

  • output – [inout] Output tensor of shape [N, H]. It computes sigmoid(input[N, :H]) x input[N, H:]

  • stream – [in] CUDA stream used for the operation.

void nvte_dglu(
const NVTETensor grad,
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the GLU activation gradient. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • grad – [in] Incoming gradient of shape [N, H].

  • input – [in] Forward input tensor of shape [N, H * 2].

  • output – [inout] Outgoing gradient of shape [N, H * 2].

  • stream – [in] CUDA stream used for the operation.

void nvte_geglu(
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the gated GeLU activation of the input. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • input – [in] Input tensor of shape [N, H * 2].

  • output – [inout] Output tensor of shape [N, H]. It computes Act(input[N, :H]) x input[N, H:]

  • stream – [in] CUDA stream used for the operation.

void nvte_swiglu(
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the gated Swish activation of the input. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • input – [in] Input tensor of shape [N, H * 2].

  • output – [inout] Output tensor of shape [N, H]. It computes Act(input[N, :H]) x input[N, H:]

  • stream – [in] CUDA stream used for the operation.

void nvte_situglu(
const NVTETensor input,
NVTETensor output,
float beta1,
float beta2,
cudaStream_t stream,
)#

Computes SiTU-GLU with configurable soft-cap parameters.

Computes beta1 * tanh(gate / beta1) * sigmoid(gate) times beta2 * tanh(up / beta2), where gate and up are the two input halves. MXFP8 output quantization is supported on SM100+.

Parameters:

  • input – [in] Input tensor of shape [N, H * 2].

  • output – [inout] Output tensor of shape [N, H].

  • beta1 – [in] Positive gate soft-cap parameter.

  • beta2 – [in] Positive up-branch soft-cap parameter.

  • stream – [in] CUDA stream used for the operation.

void nvte_clamped_swiglu(
const NVTETensor input,
NVTETensor output,
float limit,
float alpha,
cudaStream_t stream,
)#

Computes the gated Swish activation of the input used in GPT OSS.

Deprecated:

This function has been deprecated in favor of nvte_clamped_swiglu_v2, which exposes a configurable offset for the linear (gate) component. This API is preserved for backward compatibility and is equivalent to calling nvte_clamped_swiglu_v2 with glu_linear_offset = 1.0.

See openai/gpt-oss This Gated activation has two differences compared to the original SwiGLU

  1. Both gate and pre-activations are clipped based on parameter limit.

  2. Activation uses sigmoid(alpha * x) instead of sigmoid(x) used in Swish activation inspired by original GELU paper https://arxiv.org/pdf/1606.08415 If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • input – [in] Input tensor of shape [N, H * 2].

  • output – [inout] Output tensor of shape [N, H]. It computes Act(input[N, :H]) x input[N, H:]

  • limit – [in] Clipping limits for gate and pre-activation.

  • alpha – [in] Scaling factor for the sigmoid function used in the activation.

  • stream – [in] CUDA stream used for the operation.

void nvte_clamped_swiglu_v2(
const NVTETensor input,
NVTETensor output,
float limit,
float alpha,
float glu_linear_offset,
cudaStream_t stream,
)#

Computes the gated Swish activation of the input used in GPT OSS, with a configurable offset for the linear (gate) component after clamping.

See openai/gpt-oss This Gated activation has two differences compared to the original SwiGLU

  1. Both gate and pre-activations are clipped based on parameter limit.

  2. Activation uses sigmoid(alpha * x) instead of sigmoid(x) used in Swish activation inspired by original GELU paper https://arxiv.org/pdf/1606.08415 If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • input – [in] Input tensor of shape [N, H * 2].

  • output – [inout] Output tensor of shape [N, H]. It computes Act(input[N, :H]) x (input[N, H:] + glu_linear_offset)

  • limit – [in] Clipping limits for gate and pre-activation.

  • alpha – [in] Scaling factor for the sigmoid function used in the activation.

  • glu_linear_offset – [in] Offset added to the linear component after clamping (typically 1.0).

  • stream – [in] CUDA stream used for the operation.

void nvte_scaled_swiglu(
const NVTETensor input,
const NVTETensor act_scales,
NVTETensor output,
int64_t glu_interleave_size,
cudaStream_t stream,
)#

Computes ScaledSwiGLU without materializing GLU deinterleave.

Computes output = SwiGLU(input) * act_scales[:, None]. If glu_interleave_size > 0, input is interpreted as interleaved [activation_block, linear_block] chunks of that size.

Parameters:

  • input – [in] Input tensor of shape [N, H * 2].

  • act_scales – [in] Row-wise activation scales of shape [N].

  • output – [inout] Output tensor of shape [N, H].

  • glu_interleave_size – [in] 0 for non-interleaved layout; otherwise a positive multiple of 32 that divides H.

  • stream – [in] CUDA stream used for the operation.

void nvte_scaled_situglu(
const NVTETensor input,
const NVTETensor act_scales,
NVTETensor output,
float beta1,
float beta2,
int64_t glu_interleave_size,
cudaStream_t stream,
)#

Computes row-scaled SiTU-GLU without materializing GLU deinterleave.

Parameters:

  • input – [in] Input tensor of shape [N, H * 2].

  • act_scales – [in] Row-wise activation scales of shape [N].

  • output – [inout] Output tensor of shape [N, H].

  • beta1 – [in] Positive gate soft-cap parameter.

  • beta2 – [in] Positive up-branch soft-cap parameter.

  • glu_interleave_size – [in] 0 for contiguous layout; otherwise a positive multiple of 32 that divides H.

  • stream – [in] CUDA stream used for the operation.

void nvte_scaled_clamped_swiglu(
const NVTETensor input,
const NVTETensor act_scales,
NVTETensor output,
float limit,
float alpha,
float glu_linear_offset,
int64_t glu_interleave_size,
cudaStream_t stream,
)#

Computes ScaledClampedSwiGLU without materializing GLU deinterleave.

Computes output = ClampedSwiGLU(input) * act_scales[:, None]. This uses the same clamping, alpha, and linear-offset semantics as nvte_clamped_swiglu_v2.

Parameters:

  • input – [in] Input tensor of shape [N, H * 2].

  • act_scales – [in] Row-wise activation scales of shape [N].

  • output – [inout] Output tensor of shape [N, H].

  • limit – [in] Clipping limit.

  • alpha – [in] Activation sigmoid alpha.

  • glu_linear_offset – [in] Offset added to linear component after clamping.

  • glu_interleave_size – [in] 0 for non-interleaved layout; otherwise a positive multiple of 32 that divides H.

  • stream – [in] CUDA stream used for the operation.

void nvte_reglu(
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the gated ReLU activation of the input. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • input – [in] Input tensor of shape [N, H * 2].

  • output – [inout] Output tensor of shape [N, H]. It computes Act(input[N, :H]) x input[N, H:]

  • stream – [in] CUDA stream used for the operation.

void nvte_qgeglu(
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the gated Quick GeLU activation of the input. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • input – [in] Input tensor of shape [N, H * 2].

  • output – [inout] Output tensor of shape [N, H]. It computes Act(input[N, :H]) x input[N, H:]

  • stream – [in] CUDA stream used for the operation.

void nvte_sreglu(
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the gated Squared ReLU activation of the input. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • input – [in] Input tensor of shape [N, H * 2].

  • output – [inout] Output tensor of shape [N, H]. It computes Act(input[N, :H]) x input[N, H:]

  • stream – [in] CUDA stream used for the operation.

void nvte_dgeglu(
const NVTETensor grad,
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the gated GeLU activation gradient. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • grad – [in] Incoming gradient of shape [N, H].

  • input – [in] Forward input tensor of shape [N, H * 2].

  • output – [inout] Outgoing gradient of shape [N, H * 2].

  • stream – [in] CUDA stream used for the operation.

void nvte_dswiglu(
const NVTETensor grad,
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the gated Swish activation gradient. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • grad – [in] Incoming gradient of shape [N, H].

  • input – [in] Forward input tensor of shape [N, H * 2].

  • output – [inout] Outgoing gradient of shape [N, H * 2].

  • stream – [in] CUDA stream used for the operation.

void nvte_dsituglu(
const NVTETensor grad,
const NVTETensor input,
NVTETensor output,
float beta1,
float beta2,
cudaStream_t stream,
)#

Computes the SiTU-GLU activation gradient.

Parameters:

  • grad – [in] Incoming gradient of shape [N, H].

  • input – [in] Forward input tensor of shape [N, H * 2].

  • output – [inout] Outgoing gradient of shape [N, H * 2].

  • beta1 – [in] Positive gate soft-cap parameter.

  • beta2 – [in] Positive up-branch soft-cap parameter.

  • stream – [in] CUDA stream used for the operation.

void nvte_clamped_dswiglu(
const NVTETensor grad,
const NVTETensor input,
NVTETensor output,
float limit,
float alpha,
cudaStream_t stream,
)#

Computes the gradient of gated Swish activation of the input used in GPT OSS.

Deprecated:

This function has been deprecated in favor of nvte_clamped_dswiglu_v2, which exposes a configurable offset for the linear (gate) component. This API is preserved for backward compatibility and is equivalent to calling nvte_clamped_dswiglu_v2 with glu_linear_offset = 1.0.

openai/gpt-oss This activation has two differences compared to the original SwiGLU

  1. Both gate and pre-activations are clipped based on parameter limit.

  2. Activation uses sigmoid(alpha * x) instead of sigmoid(x) used in Swish activation inspired by original GELU paper https://arxiv.org/pdf/1606.08415 If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • grad – [in] Incoming gradient of shape [N, H].

  • input – [in] Forward input tensor of shape [N, H * 2].

  • output – [inout] Outgoing gradient of shape [N, H * 2].

  • limit – [in] Clipping limits for gate and pre-activation.

  • alpha – [in] Scaling factor for the sigmoid function used in the activation.

  • stream – [in] CUDA stream used for the operation.

void nvte_clamped_dswiglu_v2(
const NVTETensor grad,
const NVTETensor input,
NVTETensor output,
float limit,
float alpha,
float glu_linear_offset,
cudaStream_t stream,
)#

Computes the gradient of gated Swish activation of the input used in GPT OSS, with a configurable offset for the linear (gate) component after clamping.

openai/gpt-oss This activation has two differences compared to the original SwiGLU

  1. Both gate and pre-activations are clipped based on parameter limit.

  2. Activation uses sigmoid(alpha * x) instead of sigmoid(x) used in Swish activation inspired by original GELU paper https://arxiv.org/pdf/1606.08415 If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • grad – [in] Incoming gradient of shape [N, H].

  • input – [in] Forward input tensor of shape [N, H * 2].

  • output – [inout] Outgoing gradient of shape [N, H * 2].

  • limit – [in] Clipping limits for gate and pre-activation.

  • alpha – [in] Scaling factor for the sigmoid function used in the activation.

  • glu_linear_offset – [in] Offset added to the linear component after clamping (typically 1.0).

  • stream – [in] CUDA stream used for the operation.

void nvte_scaled_dswiglu(
const NVTETensor grad,
const NVTETensor input,
const NVTETensor act_scales,
NVTETensor grad_input,
NVTETensor grad_act_scales,
int64_t glu_interleave_size,
cudaStream_t stream,
)#

Computes ScaledSwiGLU backward without materializing GLU deinterleave.

The optional grad_act_scales tensor may be null. When present, it receives sum(dY * SwiGLU(input), dim=-1).

Parameters:

  • grad – [in] Incoming gradient of shape [N, H].

  • input – [in] Forward input tensor of shape [N, H * 2].

  • act_scales – [in] Row-wise activation scales of shape [N].

  • grad_input – [inout] Outgoing gradient of shape [N, H * 2].

  • grad_act_scales – [inout] Optional row-wise scale gradient of shape [N], or null.

  • glu_interleave_size – [in] 0 for non-interleaved layout; otherwise a positive multiple of 32 that divides H.

  • stream – [in] CUDA stream used for the operation.

void nvte_scaled_dsituglu(
const NVTETensor grad,
const NVTETensor input,
const NVTETensor act_scales,
NVTETensor grad_input,
NVTETensor grad_act_scales,
float beta1,
float beta2,
int64_t glu_interleave_size,
cudaStream_t stream,
)#

Computes row-scaled SiTU-GLU backward without materializing GLU deinterleave.

When grad_act_scales is non-null, it receives the reduction sum(dY * SiTUGLU(input), dim=-1).

Parameters:

  • grad – [in] Incoming gradient of shape [N, H].

  • input – [in] Forward input tensor of shape [N, H * 2].

  • act_scales – [in] Row-wise activation scales of shape [N].

  • grad_input – [inout] Outgoing gradient of shape [N, H * 2].

  • grad_act_scales – [inout] Optional row-wise scale gradient of shape [N], or null.

  • beta1 – [in] Positive gate soft-cap parameter.

  • beta2 – [in] Positive up-branch soft-cap parameter.

  • glu_interleave_size – [in] 0 for contiguous layout; otherwise a positive multiple of 32 that divides H.

  • stream – [in] CUDA stream used for the operation.

void nvte_scaled_clamped_dswiglu(
const NVTETensor grad,
const NVTETensor input,
const NVTETensor act_scales,
NVTETensor grad_input,
NVTETensor grad_act_scales,
float limit,
float alpha,
float glu_linear_offset,
int64_t glu_interleave_size,
cudaStream_t stream,
)#

Computes ScaledClampedSwiGLU backward without materializing GLU deinterleave.

The optional grad_act_scales tensor may be null. When present, it receives sum(dY * ClampedSwiGLU(input), dim=-1).

Parameters:

  • grad – [in] Incoming gradient of shape [N, H].

  • input – [in] Forward input tensor of shape [N, H * 2].

  • act_scales – [in] Row-wise activation scales of shape [N].

  • grad_input – [inout] Outgoing gradient of shape [N, H * 2].

  • grad_act_scales – [inout] Optional row-wise scale gradient of shape [N], or null.

  • limit – [in] Clipping limit.

  • alpha – [in] Activation sigmoid alpha.

  • glu_linear_offset – [in] Offset added to linear component after clamping.

  • glu_interleave_size – [in] 0 for non-interleaved layout; otherwise a positive multiple of 32 that divides H.

  • stream – [in] CUDA stream used for the operation.

void nvte_dreglu(
const NVTETensor grad,
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the gated ReLU activation gradient. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • grad – [in] Incoming gradient of shape [N, H].

  • input – [in] Forward input tensor of shape [N, H * 2].

  • output – [inout] Outgoing gradient of shape [N, H * 2].

  • stream – [in] CUDA stream used for the operation.

void nvte_dqgeglu(
const NVTETensor grad,
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the gated Quick GeLU activation gradient. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • grad – [in] Incoming gradient of shape [N, H].

  • input – [in] Forward input tensor of shape [N, H * 2].

  • output – [inout] Outgoing gradient of shape [N, H * 2].

  • stream – [in] CUDA stream used for the operation.

void nvte_dsreglu(
const NVTETensor grad,
const NVTETensor input,
NVTETensor output,
cudaStream_t stream,
)#

Computes the gated Squared ReLU activation gradient. If the scaling mode of the output tensor is set to NVTE_MXFP8_1D_SCALING, the block quantization (MXFP8) of the specified shape of the block will be used.

Parameters:

  • grad – [in] Incoming gradient of shape [N, H].

  • input – [in] Forward input tensor of shape [N, H * 2].

  • output – [inout] Outgoing gradient of shape [N, H * 2].

  • stream – [in] CUDA stream used for the operation.

void nvte_scaled_srelu(
const NVTETensor input,
const NVTETensor act_scales,
NVTETensor output,
cudaStream_t stream,
)#

Computes ScaledSReLU.

Computes output = SReLU(input) * act_scales[:, None].

Parameters:

  • input – [in] Input tensor for activation.

  • act_scales – [in] Row-wise activation scales of shape [N].

  • output – [inout] Output tensor.

  • stream – [in] CUDA stream used for the operation.

void nvte_scaled_dsrelu(
const NVTETensor grad,
const NVTETensor input,
const NVTETensor act_scales,
NVTETensor grad_input,
NVTETensor grad_act_scales,
cudaStream_t stream,
)#

Computes ScaledSReLU backward.

The optional grad_act_scales tensor may be null. When present, it receives sum(dY * SReLU(input), dim=-1).

Parameters:

  • grad – [in] Incoming gradient.

  • input – [in] Forward input tensor.

  • act_scales – [in] Row-wise activation scales of shape [N].

  • grad_input – [inout] Outgoing input gradient.

  • grad_act_scales – [inout] Optional row-wise scale gradient of shape [N], or null.

  • stream – [in] CUDA stream used for the operation.