Skip to visualization
ā† visualization / MinGPT with Attention

MinGPT with Attention

plain causal transformer
FOUNDATION
iT=8

šŸ” Variable Inspector

click a mapped line
Select a block or mapped code line.
About this lesson
Quick answer

What MinGPT with Attention explains

A minimal decoder-only Transformer with plain causal attention, learned position lookup, residual blocks, and next-token prediction.

Key concepts

Primary research

Attention Is All You Need Ā· Vaswani et al.