The Tiny LLM Lab only looks at the last token. Real LLMs use a transformer, where every token can look back at all the tokens before it. This is a real one, small enough to see inside, trained in your browser.
We add one sentence to the Tiny LLM Lab's text: "Ruby likes River." Now the word after "likes" depends on who is doing the liking, which the last word alone can't tell you.
Click any block, or step through them in order. Every number shown is computed live by this model for your prompt.
Same blocks, same training rule. Real models are bigger in every direction and stack the transformer block many times.
A simplified model for understanding: one transformer block with one attention head, and no layer normalization. Real transformers use many heads per block, layer normalization, and dozens of stacked blocks, but every block does the same two jobs shown here: attention moves information between tokens, and the feed-forward network processes it inside each token. Next: how one of these weights is made.