The rain kept falling, and the city began to
your cursor is the temperature — move it →
The machine that finishes your sentence
A slow-motion look at how a language model chooses one word.
Every reply from a language model begins the same way: a machine holds an unfinished sentence and asks a single question — what comes next? Not what is true, not what is meant. Just: next.
I wanted to watch that moment properly, so I took one ordinary sentence and slowed it down about a thousand times. Everything below is that one question being answered once.
Scroll gently. The sentence stays with you the whole way down — and it will answer your cursor as readily as mine. One honesty note: every number on this page (token IDs, coordinates, weights, percentages) is a toy value chosen to teach the shape, not a reading from a real model.
You're reading the pocket version — everything works under a thumb, but on a desk the whole page answers your cursor. Worth a second visit.
First, the words stop being words
A tokenizer cuts the sentence into pieces it has met before. Common words tend to survive whole; rarer ones split into fragments. Notice falling — in this toy example it travels as two shards.
From here on, the model never sees letters again. It works with the numbers underneath.
Go deeper: the tax on every other language →Each piece trades its spelling for a position
Every token becomes a long column of numbers — coordinates in a space the model learned during training. Words that behave alike tend to sit near each other there.
Rain lives closer to weather than to invoice. Meaning, at this depth, is geometry.
The pieces look at each other
For the empty slot at the end, the model asks: which earlier pieces matter right now? Those are the arcs. To leans hardest on began and city, faintly on the rain.
Nothing is understood once and kept. The influence of every earlier piece is re-weighed at every step — the model consults its notes on the whole sentence, every single time.
Try it — hover any word and the arcs answer for itOne economy trick makes it affordable
Those notes are real: what each token contributes gets written down once — a key and a value — so the next step doesn't recompute the entire past. That ledger is the KV-cache.
It isn't long-term memory. It's a receipt.
In the works: a million tokens is not a memoryThen, the whole point: a ranked list
Every token the model knows, scored for how well it would continue the sentence. This is not a decision yet — it's a distribution. The model is never certain; it is only ever weighted.
The dice-throw that follows is called sampling, and how loose the dice are has a name: temperature. (Some systems simply take the top of the list, every time — that is temperature zero.)
Try it — drag the temperature, then roll the dice Go deeper: the architecture of confidence →And then it happens again
The token you rolled joins the sentence, the ledger grows by one line, and everything you just watched runs again — for the next word, and the next.
A reply is never written. It accrues, one weighted guess at a time. That is the entire machine.
−1.077
0.482
⋮
0.006 rain110401.334
0.091
−0.750
⋮
0.412 kept6524−0.118
0.966
0.204
⋮
−0.533 fall44980.871
−0.402
1.190
⋮
0.088 ing,278−0.940
0.315
0.577
⋮
−0.201 and3230.104
0.729
−0.866
⋮
0.350 the2620.198
−1.041
0.455
⋮
0.019 city1748−0.622
1.208
0.340
⋮
0.914 began25400.455
0.038
−1.310
⋮
0.267 to284−0.077
0.512
0.883
⋮
−0.446 glow