← All demos
k_ai-basics · Demo 05 · Context window

Why AI forgets

A notepad that only fits so much

Imagine telling a friend a long story. Your friend writes it on a tiny notepad that only fits a few lines. When the notepad is full and you say something new, your friend rubs out the oldest line to make room. Later, if you ask "What did I say at the start?", it is gone. Your friend didn't choose to forget. The notepad was just full.

AI chatbots work the same way. They can only "see" a certain number of tokens at once. Tokens are the small pieces words are cut into (see Demo 01). This limit is called the context window. When a chat gets too long, the oldest messages fall off the notepad.

How are tokens counted? Each message is cut into tokens the same way GPT-2 does it in Demo 01, and then the pieces are counted. For example, "Hi! My name is Priya." is 8 tokens:

  1. Hi17250
  2. !0
  3. ·My2011
  4. ·name1438
  5. ·is318
  6. ·Pri4389
  7. ya3972
  8. .13

Roughly, a common word is 1 token, and marks like ! and . count too. A name GPT-2 rarely saw, like Priya, is cut into smaller pieces: "·Pri" + "ya". Other languages and emoji take even more. (· marks a space, and the small number is each token's ID.)

This page gives the AI a tiny notepad on purpose: just 30 tokens. That way, you can watch it fill up and see which message gets rubbed out.

Watch the notepad fill up

4 messages, one after another, on a notepad of 30 tokens. The bar shows how full it is after each one.

  1. You: Hi! My name is Priya. +8 tokens

    8/30 8/30 tokens

  2. You: I live in Hyderabad with my cat. +9 tokens

    17/30 17/30 tokens

  3. You: My favourite food is chicken biryani. +9 tokens

    26/30 26/30 tokens

  4. You: What is my name? +5 tokens

    ✗ Rubbed out: Hi! My name is Priya. (8 tokens)

    23/30 23/30 tokens

What the AI can still see now:

  • I live in Hyderabad with my cat. (9 tokens)
  • My favourite food is chicken biryani. (9 tokens)
  • What is my name? (5 tokens)

How is a number like "+8 tokens" worked out? See how tokens are counted, at the top.

❌ Just as you ask "What is my name?", the message with your name falls off. The AI isn't being rude. It simply can't see that message any more.

The lesson: an AI only remembers what still fits in its context window, and it always loses the oldest part first. Today's AIs have huge notepads, with hundreds of thousands of tokens, so this happens much later. But it still happens in very long chats. That is why an AI can lose track of something you said early on.

Try it yourself

Send messages and watch the notepad. Tap a message below to send it, or type your own and press Enter. Want a bigger notepad? Change its size. That starts a new chat.

Getting the notepad ready in your browser…

Everything on this page runs on your own device. Nothing you type is sent anywhere, and nothing is saved. This page never talks to an AI. It only counts tokens using GPT-2's list, and follows the demo's forgetting rule.

Curious how it works? The Python code behind this page is on GitHub, where you can run the same notepad in a terminal.

See the code on GitHub Run it in a terminal instead