SinusoidalEmbedding

class sdm.nn.SinusoidalEmbedding(theta: float = 10000)

Bases: Module

Sinusoidal positional embedding from the “Attention Is All You Need” paper.

Parameters:

theta (float) – The base used to compute positional frequencies.

forward(x: Tensor, pos: Tensor | None = None) → Tensor

The forward pass.

Parameters:

• x (Tensor) – Tensor with shape [..., S, C], where S is the sequence length and C is the number of channels.

• pos (Tensor | None) – Optional position indices with shape [..., S].

Return type:

Tensor