Skip to visualization
← visualization / Sinusoidal PE

minGPT · Sinusoidal PE

fixed sin/cos, added at input
Sinusoidal (Sin/Cos)
i
i
i
iT=8
i

🔍 Variable Inspector

click any code line
Click any line of code to inspect its variables, shapes, and meanings.
About this lesson
Quick answer

What Sinusoidal Positional Encoding explains

Fixed sine and cosine frequencies give token vectors an explicit, non-learned sense of position.

Key concepts

Primary research

Attention Is All You Need · Vaswani et al.