A real, very small language model that runs in your browser. It learns which word comes next from any text you give it, saves what it learned as numbers, and uses those frozen numbers to predict.
Tokens and their IDs
Training examples (current token → next token)
rows = current token · columns = possible next token
Training happens in memory. Saving writes the learned numbers into a weights file. Prediction only ever uses the saved file.
No saved model yet. Train, then save.
Same format as tiny_llm.py, so a model trained in Python can be loaded here.
The saved weights are loaded and frozen. Nothing you do here changes them.
Raw scores for every token in the vocabulary.
Higher score, higher chance. All chances add up to 100%.
Temperature changes the jar: low makes the top token fill almost every marble, high makes the colours more even. Top-k decides how many different colours are allowed in the jar at all; top-k 1 means only the top token, so it is picked every time.
A simplified model for understanding. Its tokens are whole words, it looks only at the last token, and it uses one grid of weights. Large language models use subword tokens, billions of weights and the whole input to compute their scores, but follow the same cycle: random weights, training that adjusts them, a saved weights file, and frozen weights that score every possible next token. Next: how text becomes numbers.