SinusoidalEmbedding

class sdm.nn.SinusoidalEmbedding(theta: float = 10000)

Bases: Module

Sinusoidal positional embedding from the “Attention Is All You Need” paper.

Parameters:

theta (float) – The base used to compute positional frequencies.

forward(x: Tensor) → Tensor

The forward pass.

Parameters:

x (Tensor) – Tensor with shape [..., S, C], where S is the sequence length and C is the number of channels.

Return type:

Tensor