convs2s

attention_wn_layer

Implementation of the attention layer for convs2s. Inspired from https://github.com/tobyyouup/conv_seq2seq

class parts.convs2s.attention_wn_layer.AttentionLayerNormalized(in_dim, embed_size, layer_id, add_res, mode, scaling_factor=0.7071067811865476, normalization_type='weight_norm', regularizer=None, init_var=None)[source]

Bases: tensorflow.python.layers.base.Layer

Attention layer for convs2s with weight normalization

__init__(in_dim, embed_size, layer_id, add_res, mode, scaling_factor=0.7071067811865476, normalization_type='weight_norm', regularizer=None, init_var=None)[source]

initializes the attention layer. It uses weight normalization for linear projections (Salimans & Kingma, 2016) w = g * v/2-norm(v)

Parameters:
  • in_dim – int last dimension of the inputs
  • embed_size – int target embedding size
  • layer_id – int the id of current convolution layer
  • add_res – bool whether residual connection should be added or not
  • mode – str current mode
call(input, target_embed, encoder_output_a, encoder_output_b, input_attention_bias)[source]

Calculates the attention vectors.

Parameters:
  • input – A float32 tensor with shape [batch_size, length, in_dim]
  • target_embed – A float32 tensor with shape [batch_size, length, in_dim] containing the target embeddings
  • encoder_output_a – A float32 tensor with shape [batch_size, length, out_dim] containing the first encoder outputs, uses as the keys
  • encoder_output_b – A float32 tensor with shape [batch_size, length, src_emb_dim] containing the second encoder outputs, uses as the values
  • input_attention_bias – A float32 tensor with shape [batch_size, length, 1] containing the bias used to mask the paddings
Returns:

float32 tensor with shape [batch_size, length, out_dim].

conv_wn_layer

Implementation of a 1d convolutional layer with weight normalization. Inspired from https://github.com/tobyyouup/conv_seq2seq

class parts.convs2s.conv_wn_layer.Conv1DNetworkNormalized(in_dim, out_dim, kernel_width, mode, layer_id, hidden_dropout, conv_padding, decode_padding, activation=<function gated_linear_units>, normalization_type='weight_norm', regularizer=None, init_var=None)[source]

Bases: tensorflow.python.layers.base.Layer

1D convolutional layer with weight normalization

__init__(in_dim, out_dim, kernel_width, mode, layer_id, hidden_dropout, conv_padding, decode_padding, activation=<function gated_linear_units>, normalization_type='weight_norm', regularizer=None, init_var=None)[source]

initializes the 1D convolution layer. It uses weight normalization (Salimans & Kingma, 2016) w = g * v/2-norm(v)

Parameters:
  • in_dim – int last dimension of the inputs
  • out_dim – int new dimension for the output
  • kernel_width – int width of kernel
  • mode – str the current mode
  • layer_id – int the id of current convolution layer
  • hidden_dropout – float the keep-dropout value used on the input. Give 1.0 if no dropout. It is used to initialize the weights of convolution.
  • conv_padding – str the type of padding done for convolution
  • decode_padding – bool specifies if this convolution layer is in decoder or not in decoder padding is done explicitly before convolution
  • activation – the activation function applies after the convolution
  • normalization_type – str specifies the normalization used for the layer. “weight_norm” for weight normalization or “batch_norm” for batch normalization or “layer_norm” for layer normalization
  • regularizer – the regularizer for the batch normalization
call(input)[source]

Applies convolution with gated linear units on x.

Parameters:x – A float32 tensor with shape [batch_size, length, in_dim]
Returns:float32 tensor with shape [batch_size, length, out_dim].

ffn_wn_layer

Implementation of fully connected network with weight normalization. Inspired from https://github.com/tobyyouup/conv_seq2seq

class parts.convs2s.ffn_wn_layer.FeedFowardNetworkNormalized(in_dim, out_dim, dropout, var_scope_name, mode, normalization_type='weight_norm', regularizer=None, init_var=None)[source]

Bases: tensorflow.python.layers.base.Layer

Fully connected feedforward network with weight normalization

__init__(in_dim, out_dim, dropout, var_scope_name, mode, normalization_type='weight_norm', regularizer=None, init_var=None)[source]

initializes the linear layer. This layer projects from in_dim-dimenstional space to out_dim-dimentional space. It uses weight normalization (Salimans & Kingma, 2016) w = g * v/2-norm(v)

Parameters:
  • in_dim – int last dimension of the inputs
  • out_dim – int new dimension for the output
  • dropout – float the keep-dropout value used in the previous layer. It is used to initialize the weights. Give 1.0 if no dropout.
  • var_scope_name – str the scope name for the weight variables
  • mode – str current mode
  • normalization_type – str specifies the normalization used for this layer. “weight_norm” for weight normalization or “batch_norm” for batch normalization
call(x)[source]

Projects x with its linear transformation.

Parameters:x – A float32 tensor with shape [batch_size, length, in_dim]
Returns:float32 tensor with shape [batch_size, length, out_dim].