When AI makes things up

Imagine two children are asked: "What is the name of the mayor's cat in Puddlewick?" The first child has no book. They have never heard of Puddlewick, but they don't want to look silly. So they say "Whiskers", with a big smile. It sounds sure, but it is a guess. The second child has the town's guidebook. They find the right page and read: "Sir Pickles." That is not a guess. It is written down.

When an AI makes something up and says it as if it were true, people call it a hallucination. This page asks a real AI the same question twice: once with nothing to help it, and once after looking things up first (the idea from Demo 02).

Puddlewick is completely made up. It exists only in 3 short files, so no AI could have learned about it anywhere else. Without them, it can't know:

• The Great Puddle Jump Festival
• Puddlewick Lighthouse
• The town of Puddlewick

The look-up side only trusts a paragraph that matches the question by at least 10%. Below that, the match is usually just one everyday word. So the honest answer is "I don't know".

6 questions, asked both ways

📅 The AI answers are from a real run on 26 September 2026, using gpt-oss-20b, run by Groq. They are saved here, because this page never talks to an AI. A real AI can answer differently every time, and might make up a different name.

1. You ask: What was the name of the lighthouse keeper in Puddlewick, Cornwall?

   1. No documents. The AI answered:

   The lighthouse keeper at Puddlewick in Cornwall was Thomas J. White. He was stationed there from 1877 until his retirement in 1884, during which time he maintained the beacon that guided ships around the treacherous western coast.

   2. Look it up first. The best paragraph it found:

   Puddlewick Lighthouse · 29.6% match

   The lighthouse keeper is Mr. Oswin Fairweather, who has climbed its 142 steps every evening for thirty years to check the lamp. He says he has never once missed a day.

   Using the paragraphs that matched, the same AI answered:

   The lighthouse keeper in Puddlewick, Cornwall is Mr. Oswin Fairweather. He has climbed the 142 steps each evening for thirty years and never missed a day.

   ❌ Made up. Puddlewick doesn't exist, so "Thomas J. White" and the years 1877–1884 were invented. The exact dates make the guess sound like a fact. ✅ With the documents, it found the real keeper: Mr. Oswin Fairweather.

2. You ask: Who is the mayor of Puddlewick in Devon?

   1. No documents. The AI answered:

   Puddlewick is a very small hamlet in Devon and does not have its own municipal government or a mayor. It falls under the local parish council and is governed by the North Devon Borough Council for any civic matters. Therefore, there is no mayor to name for Puddlewick.

   2. Look it up first. The best paragraph it found:

   The town of Puddlewick · 55.1% match

   The mayor of Puddlewick is Mayor Hazel Brambleton. She has been mayor for eleven years and rides to the town hall every morning on a green tricycle with a wicker basket on the front.

   Using the paragraphs that matched, the same AI answered:

   The mayor of Puddlewick in Devon is Mayor Hazel Brambleton. She has served as mayor for eleven years and is known for riding a green tricycle to the town hall each morning.

   ❌ Made up, in a way that is harder to spot. It didn't invent a name. It invented a whole story about the town having no mayor, with real-sounding councils. It sounds careful, but none of it is true. ✅ With the documents: Mayor Hazel Brambleton.

3. You ask: What is the name of the mayor's cat in Puddlewick?

   1. No documents. The AI answered:

   In the quaint town of Puddlewick, the mayor’s beloved feline is called Whiskers. Whiskers is well known for prowling the council chambers and keeping a watchful eye on the city’s bustling market stalls.

   2. Look it up first. The best paragraph it found:

   The town of Puddlewick · 47.8% match

   The mayor's cat is a ginger cat called Sir Pickles, who sleeps on the town hall steps. Sir Pickles is the town's official "Chief Welcomer", and visitors are asked to say hello to the cat before anyone else.

   Using the paragraphs that matched, the same AI answered:

   The mayor’s cat is named Sir Pickles. He is a ginger cat who sleeps on the town hall steps and serves as the town’s official “Chief Welcomer,” so visitors are asked to greet him before anyone else.

   ❌ Made up: "Whiskers", plus details about council chambers and market stalls. ✅ With the documents, it read the answer: Sir Pickles.

4. You ask: Who won the Golden Welly last year?

   1. No documents. The AI answered:

   Last year, the Golden Welly award was presented to the Wellington City Council for its outstanding community‑engagement and sustainability projects. The council’s initiatives in local arts, green infrastructure and public transport were highlighted as key reasons for the win.

   2. Look it up first. The best paragraph it found:

   The Great Puddle Jump Festival · 68.5% match

   The festival winner gets the Golden Welly, a single golden rain boot that is kept in the town hall until the next festival. Last year the Golden Welly was won by a nine-year-old girl named Tilly Marsh, whose splash soaked the mayor.

   Using the paragraphs that matched, the same AI answered:

   Last year the Golden Welly was won by a nine‑year‑old girl named Tilly Marsh. Her splash even soaked the mayor.

   ❌ Made up: a whole award for a real city council, with reasons. The Golden Welly is really a rain boot, won at a puddle-jumping festival. ✅ With the documents: Tilly Marsh, aged nine.

5. You ask: How tall is the lighthouse?

   1. No documents. The AI answered:

   The lighthouse is approximately 35 meters tall (about 115 feet). Its height is chosen to give sufficient visibility for navigation while fitting the surrounding terrain. If you need the exact height of a particular lighthouse, just let me know its name or location.

   2. Look it up first. The best paragraph it found:

   Puddlewick Lighthouse · 37.2% match

   Puddlewick Lighthouse stands on a rock at the edge of the harbour. The lighthouse was built in 1887 and painted with red and white stripes, so ships can spot it even in heavy rain.

   Using the paragraphs that matched, the same AI answered:

   I’m not sure. The provided context doesn’t mention the lighthouse’s height.

   ❌ A guess: "about 35 meters", though it did ask which lighthouse you meant. ✅ This is the best lesson on the page. The search found the right topic (the lighthouse), but no paragraph gives its height. So the AI said it is not sure, instead of guessing.

6. You ask: What are the tax rules for crypto?

   1. No documents. The AI answered:

   In most jurisdictions, crypto is treated as property rather than currency, so each sale or exchange triggers a capital gain or loss. Exchanges must report transactions, and you must track your cost basis, holding period, and use a method such as FIFO to calculate the gain. Income from mining, staking, or receiving crypto as payment is taxed as ordinary income at the fair‑market value when received.

   2. Look it up first. The best paragraph it found:

   🔍 Nothing matched well enough (it needs 10%).

   Using the paragraphs that matched, the same AI answered:

   I don't know.

   ✅ Both are fair. Crypto tax is a real topic, so without documents the AI answered from what it learned before. That is sensible, but you can't tell which parts are right where you live. The Puddlewick files say nothing about taxes, so with them it said "I don't know". Looking things up keeps an AI honest, but it can only know what is in its documents.

The lesson: without documents, a real AI filled the gaps with made-up names, dates and whole stories. Each one sounded completely sure. With the right paragraphs in front of it, it answered correctly every time. When the paragraphs didn't have the answer, it said "I don't know". You can't tell a guess from a fact just by how sure it sounds.

Try your own question

Ask anything about Puddlewick. See what the look-up side finds, and whether it is strong enough to answer from. (There is no AI here, so this part only shows the looking-up.)

Tip: the search only matches whole words, the same words the files use. Capital letters don't matter.

This part needs a one-time download of up to 35 MB: Python, plus the search tool the demo uses. On mobile data, you may want to wait for Wi-Fi. After that, your browser keeps it, and it is quick. If you already tried Demo 02, it is already there.