How do language models know, remember and reason? They don't.

On 3 November I gave a talk at Alex’s AI meetup at REALITY in Da Nang. It was for people who use ChatGPT every day and have no idea what’s inside it.

The whole talk is three questions. How does it know things? How does it remember things? How does it reason? The answer to all three is: it doesn’t.

The full deck is below, so you can click through it yourself.

Watch it

What the talk covers

• “AI” is old. Deep Blue beat Kasparov in 1997. What’s new is language models.
• “Claude” and “GPT” are families of models, not models. Opus 5.5 is a model. Closed weights vs open weights.
• Text becomes numbers. There’s a live tokenizer demo (the same one GPT-4o uses), and the same string always maps to the same numbers.
• The core problem: what do you multiply the input numbers by to get a sensible answer back? A toy model that just doubles everything produces gibberish.
• A neural net is just numbers (weights) on lines, and nobody sets them by hand.
• Training: guess, measure how wrong you were, nudge every weight a tiny bit, repeat billions of times. The live net shows the error line falling.
• Billions of weights need trillions of examples, which is why they took the whole internet.
• Images work the same way. A picture is a grid of pixels, each pixel is three numbers (red, green, blue), flattened into one long list.
• What you actually download from Hugging Face is a 724 MB file of 362 million decimal numbers. No code, no facts, no memory.
• So it doesn’t know (it predicts likely text), it doesn’t remember (the whole chat gets re-sent every turn), and it doesn’t reason (it predicts text that looks like reasoning).
• The software around the model (the harness) fakes all three: tools to look things up, memory files put back into the prompt, loops that check its own work.
• What I use my harness for: driving my phone and my Quest, reverse-engineered smart home apps, floor plans from my Roborock, AI phone calls, banking behind confirm gates.

From the room

It ran way longer than planned, mostly because people kept asking questions. I wanted to nerd out about the harness and first principles. The room wanted philosophy. Fair enough.

Thanks to Alex and Outfold Systems for hosting and filming it, and to Eric and Mikail for the photos.