←visualization / Gated DeltaNet · KDA

minGPT · Gated DeltaNet and KDA

decay plus targeted replacement

◆ KDA

n_embd

n_head

n_layer

seq_lenT=8

decay modeiPer-channel KDA

chunk sizei8

demo decayiLearned

🔍 Variable Inspector

click any code line

Click a code line to inspect its variables.

About this lesson

Quick answer

Combine global or channel-wise forgetting with targeted Delta replacement and chunk-wise recurrent-state computation.