Instance Normalization (InstanceNorm)#

API#

class warp_nn.modules.layers.InstanceNorm(
num_features: int,
*,
eps: float = 1e-05,
affine: bool = False,
initialize_parameters: bool = True,
requires_grad: bool = True,
)[source]#

Bases: GroupNorm

Apply Instance Normalization over the spatial dimensions of the input.

\[\text{InstanceNorm}(x) = \frac{x - \text{E}[x]}{\sqrt{\text{Var}[x] + \epsilon}} \, \gamma + \beta\]

The mean and the (biased) variance are computed separately for each sample and channel over the remaining (spatial) dimensions. It is equivalent to a group normalization with as many groups as channels.

Note

Unlike PyTorch’s InstanceNorm*d, running statistics are not supported: the statistics are always computed from the input data.

Learnable parameters:

Name

Shape

Description

\(\gamma\)

weight

(num_features,)

Per-channel weights. Only if affine is true

\(\beta\)

bias

(num_features,)

Per-channel bias. Only if affine is true

The weights are initialized to ones and the bias to zeros.

Parameters:

  • num_features – The number of channels expected in the input.

  • eps – The value added to the variance for numerical stability.

  • affine – Whether to include learnable per-channel affine parameters.

  • initialize_parameters – Whether to initialize the parameters with their default/initial values. If false, the parameters are left as uninitialized memory, and reading them (e.g. during a forward pass) before loading their values (e.g. from a state dictionary) is undefined behavior.

  • requires_grad – Whether the parameters and the cached output arrays of the module require gradients.

__call__(
input: array,
) → array[source]#

Forward pass of the module.

Parameters:

input – The input array, with shape (batch_size, num_features, *), with at least 3 dimensions.

Returns:

The output array, with same shape as the input array.

Raises:

ValueError – If the input array’s shape is not supported.