3. Hallucinations: Why AI Invents Things

An AI model works by predicting the most plausible next piece of text. It doesn't look things up in a database, and it has no understanding of truth — it simply produces whatever continuation sounds most likely given what you wrote. This single mechanism is what lets it summarise, translate, draft emails and answer questions. But it also means that when the model lacks the information it needs, it doesn't stop and say "I don't know." Instead, it fills the gap with something that sounds right: a fake reference, an invented statistic, a feature that doesn't exist. This is called a hallucination. It isn't a software bug that engineers can patch; it is a direct consequence of how text prediction works. The model is optimised to be plausible, not accurate. That distinction matters every time you read an AI-generated answer.

Hallucinations are especially dangerous because they look exactly like correct answers. The model uses the same confident tone whether it is right or wrong — it has no internal "certainty meter." A hallucinated legal citation will be formatted perfectly. An invented company name will sit naturally in a sentence. You cannot tell the difference by style alone; you can only tell by checking the facts. This is why the rule of thumb is: the AI proposes, a human validates, and the depth of checking should match the stakes. A brainstorming list needs a light review; a contract clause or a medical dosage needs rigorous verification.

There is also a time dimension. Every model has a knowledge cutoff — a date after which it knows nothing because its training data stops there. If you ask about a recent law, a product launched last month, or yesterday's news, the model may still answer fluently, but it is guessing. Without an external source to draw from, any answer about recent events should be treated as unreliable. Always ask yourself: could the model actually know this, or is it filling in the blank?

The main defence against hallucinations is grounding: giving the model verified material to work from instead of relying on its internal memory. In OUPI, grounding can take several forms — attaching your own documents via knowledge bases, enabling live web search, or connecting tools that return real data. A grounded answer can point to where a claim comes from; an ungrounded answer cannot. Whenever accuracy matters — figures, names, dates, legal or medical statements — ground the model and then verify the output anyway. Grounding reduces hallucinations dramatically, but it does not eliminate them entirely.