Below is one sentence with its words hidden. Slide the cover and watch an AI try to guess each word before it's shown — this is how it learns to write.
The chef added a pinch of salt, a dash of pepper, and a spoonful of glitter.
A note on the numbers · Every bar height on this page is a made-up teaching number, not a real measurement. What's real is the idea — cover a word, guess it, measure the surprise.
Try it — move across the sentence, or use arrow keys; watch which surprise-bar jumps.Try it — drag the dot; watch which surprise-bar jumps.
The word before the word
How an AI learns to write — one guessed word at a time.
Narrated by a synthetic voice — not a recording · ~0.5 MB
Here is something most people never picture: an AI is never really reading. To learn to write, it is handed a real sentence it already knows — then made to guess that sentence back, one word at a time, with everything ahead of it covered up so it can't cheat.
That guessing game is the whole engine. Millions of sentences, billions of tiny guesses — and every time the AI is caught out by a word it didn't expect, that surprise becomes a lesson. On this page you run the engine by hand, on one short, faintly absurd sentence.
I wanted to see what one of those tiny guessing games actually looks like up close, so I gave myself a control to slide through the sentence a word at a time and watch the AI sweat each one.
You're on the small-screen version — the cover still slides under your thumb, but the ranked guesses are easier to read on a wider screen.
Start where the AI starts — with almost nothing shown.
What is this guess allowed to see?
Drag the control below — or press the arrow keys — and a cover slides across the sentence. Everything to the right of it is truly hidden. When the AI is guessing the sixth word, it has never seen the seventh.
One thing to hold on to: a real AI makes all sixteen guesses at once, in a single go. Sliding through them one at a time is a lens built for your eye — not the way the machine actually works. (We come back to this at the end.)
That cover is real. Engineers call it a causal mask — a hard rule, wired into the AI, that stops any guess from peeking at words it hasn't reached yet.
Try it — drag the cover (or press the right arrow)drag the cover and watch what this guess is allowed to see.What does the AI believe before it's told the answer?
At the cover, the AI doesn't blurt out a single word. It quietly lines up every word it knows and ranks them by how likely each one feels next. The panel shows five of those words — a small handful of examples, each with a made-up score.
This is the AI's honest hunch, the instant before it's told whether it was right.
Try it — hover or taptap any earlier word to reopen the guess it faced.What does a surprise cost?
Slide past the cover and the real word appears. Now measure the surprise: how far off the AI's hunch was. If it had barely expected the real word, the surprise is large. If it had that word at the top of its list, the surprise is almost nothing.
big surprise = the AI thought that word was unlikely · engineers call this number the “loss”A hunch that nailed the word costs almost nothing. A confident guess that whiffed costs a great deal.
Try it — keep sliding; guess all sixteen and watch the row of bars fill in.A landscape made of surprises
The row of bars you've drawn is a run of small, ordinary surprises — and one tall spike, where glitter lands after a spoonful of. Nothing in the sentence prepared the AI for that word.
That spike isn't a glitch. It's exactly the kind of surprise the AI learns from: the miss nudges its own guessing — so that next time, a word like this one is a little less of a surprise. Billions of those tiny nudges, over billions of sentences, are what "learning to write" really is.
Tap any word to revisit its guess.One more thing before you leave — you did this in order. The machine didn't.
So is this how the AI reads?
No. You walked the sentence one guess at a time because your eye needed the order. The AI makes all sixteen guesses together, in one go — the flash above, every word lit at once, is closer to the truth than the slow walk you just took.
How can it be both at once and never peeking ahead? Here's the join: all sixteen guesses happen at the same instant, but each one is still only allowed to see the words before its own covered spot. Picture sixteen people standing in a row, each guessing their own hidden word at the same moment — none of them peeking past their own cover.
That's what makes learning from billions of sentences possible at all — sixteen lessons out of every single sentence, all in one go.
Look — all sixteen guesses happen together, in one go.