Recurrent Neural Network Cell (RNNCell)#

API#

class warp_nn.modules.layers.RNNCell(
input_size: int,
hidden_size: int,
*,
bias: bool = True,
initialize_parameters: bool = True,
requires_grad: bool = True,
)[source]#

Bases: Module

Apply a Elman’s Recurrent Neural Network (RNN) cell.

\[\text{RNNCell}(x, h) = \text{tanh}(W_{ih} \, x + b_{ih} + W_{hh} \, h + b_{hh})\]

Learnable parameters:

Name

Shape

Description

\(W_{ih}\)

weight_ih

(hidden_size, input_size)

Input-to-hidden weights

\(W_{hh}\)

weight_hh

(hidden_size, hidden_size)

Hidden-to-hidden weights

\(b_{ih}\)

bias_ih

(hidden_size, 1)

Input-to-hidden bias. Only if bias is true

\(b_{hh}\)

bias_hh

(hidden_size, 1)

Hidden-to-hidden bias. Only if bias is true

The parameters are initialized from the uniform distribution \(u(-k, k)\) where \(k = \frac{1}{\sqrt{\text{hidden\_size}}}\).

Parameters:

  • input_size – The number of input features.

  • hidden_size – The number of hidden features.

  • bias – Whether to include a bias term.

  • initialize_parameters – Whether to initialize the parameters with their default/initial values. If false, the parameters are left as uninitialized memory, and reading them (e.g. during a forward pass) before loading their values (e.g. from a state dictionary) is undefined behavior.

  • requires_grad – Whether the parameters and the cached output arrays of the module require gradients.

__call__(
input: array,
hidden: array,
) → array[source]#

Forward pass of the module.

Parameters:

  • input – The input array, with shape (batch_size, input_size).

  • hidden – The initial hidden state array, with shape (batch_size, hidden_size).

Returns:

The next hidden state array, with shape (batch_size, hidden_size).