←visualization / Sliding Window

minGPT · Sliding Window

local, banded causal attention

◆ SWA

n_embd

n_head

n_layer

seq_lenT=8

window Wi8

🔍 Variable Inspector

click any code line

Click a code line to inspect its variables.

About this lesson

Quick answer

Restrict each token to a local causal band. This page uses readable dense reference code and explains the optimized O(T·W) kernel separately.