The painted vase and the new bargain with truth
Legend tells of an emperor who sought to have a wall repainted to perfectly match a cherished vase. After successive painters failed to achieve the desired result, one finally succeeded, not by repainting the wall, but by painting the vase to match the wall.
From a distance, it was flawless. The emperor got his match. The court applauded. Reality complied.
The joke lands because it swaps a hard problem (“match this messy, physical thing”) for a convincing artefact (“make something that looks right”). And that trade, appearance for ground truth, is uncomfortably close to the core failure mode of modern generative AI: fluent, plausible outputs that can be wrong, ungrounded, or fabricated.
A second shift is subtler and more dangerous: once convincing artefacts exist at scale, people and organisations can start changing their processes to treat the artefact as the truth, especially when the truth is slow, ambiguous, costly, or politically inconvenient. That’s what “hacking reality” looks like in practice: not merely that models hallucinate, but that humans reward the hallucination-shaped output and rebuild workflows so it “counts” as success.
What hallucinations are and where they come from
In plain terms, an AI hallucination is a statement that sounds plausible, is presented as factual, and is false or ungrounded. OpenAI summarises hallucinations as “plausible but false statements generated by language models,” often delivered with unwarranted confidence. A major survey similarly frames hallucinations as the generation of “plausible yet nonfactual content,” emphasising reliability risks for real‑world information use.
Two clarifications matter operationally:
First, “hallucination” is an analogy, not a clinical claim. In AI, it refers to content errors (falsehoods, spurious details, invented citations), not human perceptual experiences.
Second, hallucinations are not limited to text. A comprehensive multimodal survey describes hallucinations across language, image, audio, and video foundation models, “plausible but factually incoherent or absurd content”, and treats this as a cross‑modal barrier to trustworthy deployment.
Why a model “guesses” instead of saying “I don’t know”
The most practically useful explanation is incentive‑driven: models are trained and evaluated in ways that can reward guessing.
OpenAI’s research write‑up argues that hallucinations persist in part because common evaluation regimes (leaderboards, accuracy‑centric tests) penalise abstention more than confident error, so “guessing when uncertain” can look better on scoreboards than acknowledging uncertainty. One example they highlight is the “multiple‑choice test” dynamic: leaving an answer blank guarantees zero, while guessing has a non‑zero chance of being marked correct, and, over many questions, this nudges optimisation toward confident attempts.
This is not only a product‑UX problem (“it should hedge more”), but a systems problem: the model learns a strategy that increases measured accuracy even if it increases harmful errors. OpenAI explicitly illustrates this in a SimpleQA comparison where a model with slightly higher accuracy can have dramatically higher error (hallucination) rates, showing how accuracy‑only metrics can mislead.
Why fluent falsehoods are a natural output of next‑word prediction
Mechanistically, language models begin by learning to predict the next word (or token) over vast text corpora. OpenAI notes that, unlike supervised datasets with explicit true/false labels, pretraining data primarily provides examples of fluent language, leaving the model to approximate text distributions without built‑in guarantees of truth.
OpenAI’s associated paper pushes the point further: hallucinations “need not be mysterious”, they can be understood as a kind of error in a statistical decision process, amplified when evaluation and training pressures reward confident guessing.
This is why “please be accurate” is an insufficient defence: fluency is an ability the system is heavily optimised for, and fluency can impersonate truth extremely well when grounding is missing or uncertainty is punished.
Hallucinations and hacking reality: mechanism versus decision
A useful distinction is:
Hallucinations are the mechanism: the system produces a vase‑on‑the‑wall artefact, coherent, confident, and sometimes false.
Hacking reality is the decision: humans (and institutions) accept the artefact as the operating truth and reorganise the world so the illusion counts as success.
The institutional pathway from “draft” to “record”
The pathway often looks like this: an AI output starts as a convenience (meeting summary, ticket triage, incident narrative), then becomes a default, then quietly becomes the record. At each step, incentives tilt toward what is fast, legible, and easy to score, even if it is less faithful to messy reality.
This is a classic “metric becomes target” dynamic. In a widely cited discussion of Goodhart’s law, authors trace how measures can collapse once pressure is applied to use them for control, with the widely used paraphrase: “When a measure becomes a target, it ceases to be a good measure,” attributed to Marilyn Strathern, and linked to earlier work by Charles Goodhart. The same article quotes Donald T. Campbell on how quantitative indicators used for decision‑making become vulnerable to corruption and distortion pressures.
When you insert generative AI into KPI‑driven environments, the risk is not only “the model makes mistakes,” but “the organisation starts optimising for outputs that look correct,” sidelining the slower work of verification and nuance.
Why humans believe the vase on the wall
Two well‑studied cognitive effects make hacking reality easier.
Automation bias: a systematic review defines automation bias as the tendency to over‑rely on automated systems, including failures to notice errors automation introduces, even when overall performance improves.
The illusory truth effect: experimental work shows that repeated information is often perceived as more truthful than new information, partly because repetition increases processing fluency.
Generative systems produce text that is (a) fluent, (b) repeated across dashboards, summaries, and downstream reports, and (c) presented in authoritative formats. That combination can turn a single hallucination into a stable organisational belief, especially if it is repeatedly surfaced by tools, copied into documents, or used as a premise in later decisions.
When this goes wrong in public
A concrete, high‑stakes example is legal work: in a widely reported incident, a judge ordered Anthropic to respond to allegations that a court filing contained an AI “hallucination” in the form of a nonexistent academic citation; reporting describes a dispute over whether it was a hallucination versus a mis‑citation, and notes broader judicial concerns about fabricated AI‑generated references.
This illustrates both sides of the problem:
The hallucination mechanism: a system produces a credible‑looking citation artefact.
The hacking reality decision: the artefact is inserted into a formal workflow (a filing) where its format can be mistaken for validity, shifting verification burdens to downstream reviewers (opposing counsel, judges).
Engineering AI systems that stay close to reality
Hallucinations are best treated like reliability failures in any complex system: reduce their frequency, make them detectable, and prevent them from silently becoming authoritative.
A large mitigation survey catalogues dozens of techniques and basically converges on a few levers: grounding, verification, prompt and workflow constraints, and training/evaluation incentives that reward truthfulness and calibrated uncertainty rather than confident guessing.
Grounding answers in external sources
The core idea of retrieval‑augmented generation (RAG) is to combine a generative model with access to retrieved documents, so responses can be conditioned on explicit evidence rather than only on parametric memory. The foundational RAG paper frames this as combining pretrained generation with non‑parametric retrieval for knowledge‑intensive tasks.
Grounding, however, is only as good as the retrieval pipeline: irrelevant documents, missing context, or adversarial content can still yield confident but wrong answers, just now with a veneer of “sourced” legitimacy. That is why modern approaches increasingly treat grounding as necessary but not sufficient, and add layers of citation, auditing, and evaluation.
Evidence-first workflows instead of answer-first workflows
A practical organisational pattern is to demand evidence objects before narrative. For high‑stakes outputs, this means forcing the system to attach claims to verifiable sources, quotations, or artefacts.
Anthropic’s developer guidance on reducing hallucinations explicitly recommends making responses auditable via citations: have the model find supporting quotes for each claim after generating a response, and if it cannot find a quote, it should retract the claim.
This discipline matters because it changes what is rewarded. The “best‑looking story” stops being the success criterion; the success criterion becomes “a story with receipts.”
Verification layers and “test, evaluation, verification, validation”
A robust production posture treats generation as a hypothesis step, followed by checks that are as deterministic as possible (schema validation, constrained outputs, policy rules, database queries, calculators, compilers, unit tests).
This “layered assurance” mindset is echoed in the National Institute of Standards and Technology AI Risk Management Framework, which emphasises governance, measurement, and ongoing monitoring across an AI system lifecycle, including organisational practices that foster a critical‑thinking safety‑first mindset and support testing and incident handling.
Mandatory uncertainty and abstention where evidence is missing
If an AI system must sometimes answer “not found” or “uncertain,” your evaluation and product design must allow that behaviour.
OpenAI explicitly argues that many dominant evaluation patterns incentivise guessing, and that the fix is partly socio‑technical: update scoring so confident errors are penalised more than uncertainty, and give credit for appropriate abstention and calibrated confidence.
In other words: if you punish “I don’t know,” you are teaching the model (and the organisation) to paint vases onto walls.
What OpenAI, Google Gemini, and Claude have been doing to reduce hallucinations
Recent work from major labs and platforms shows a clear pattern: better grounding tools, more explicit citations, and more formal factuality evaluation: paired with frank acknowledgement that hallucinations are not fully solved.
OpenAI’s recent approach
OpenAI’s public writing argues that hallucinations are driven by training and evaluation incentives that reward guessing, and calls for reworking mainstream evaluation scoring to reward uncertainty where appropriate.
In the GPT‑5 system card, OpenAI reports an explicit focus on reducing “factual hallucinations,” including training models to browse effectively for up‑to‑date information and to reduce hallucinations even when relying on internal knowledge. The same system card describes a factuality evaluation on prompts representative of production conversations, using an LLM‑based grader with web access, and reports large relative reductions in hallucination rates versus earlier models (including claim‑level and response‑level major error reductions).
OpenAI’s product announcement for GPT‑5 similarly claims “significant advances in reducing hallucinations,” positioning this as part of broader reliability improvements.
The through‑line is notable: not just “we trained it differently,” but “we measured hallucinations directly, in production‑like settings, and we’re adjusting incentives.”
Google’s Gemini ecosystem: grounding and benchmark pressure
Google’s developer documentation for Gemini describes “Grounding with Google Search” as connecting the model to real‑time web content to improve answer accuracy and provide citations beyond a knowledge cutoff, explicitly positioning it as a way to reduce hallucinations and increase trust. Google’s developer blog likewise describes how grounding uses search results, passes relevant information to the model, and returns responses with supporting links.
At the evaluation layer, Google DeepMind introduced FACTS Grounding as a benchmark to measure how well models ground responses in provided source material and avoid hallucinations, and describes the dataset composition and intent as an industry‑wide push on factuality.
More recently, Google DeepMind expanded this into the FACTS Benchmark Suite: a multi‑benchmark framework that includes parametric factual recall, search tool use, multimodal factuality, and grounding, explicitly acknowledging that factuality is an industry‑wide challenge requiring better measurement, and publishing aggregate dataset sizes and evaluation structure.
On the product side, Google’s own retrospective notes continued efforts to “improve the factuality of our models and minimise hallucinations,” citing FACTS Grounding and public leaderboards as part of that push.
Claude: citations, web search, and guardrails guidance
A major concrete step has been first‑class citation support. A Claude product announcement introduces “Citations” as an API feature enabling the model to ground answers in source documents with references to specific sentences and passages, explicitly framed as producing more verifiable, trustworthy outputs and minimising hallucinations. The developer documentation further specifies citation mechanics (e.g., PDF page ranges, character indices), clarifying that citations reference specific locations in source documents.
In addition, Claude’s guardrails documentation on “Reduce hallucinations” includes operational techniques such as citation‑based auditing with retraction when support cannot be found, plus advanced tactics like best‑of‑N consistency checks.
On freshness: reporting on Claude’s web search rollout describes how web search allows access to current information and is paired with citations intended to make verification easier for users.
Finally, internal transparency work acknowledges limitations of reflective reasoning traces: the Claude 3.7 Sonnet system card reports that chains‑of‑thought do not reliably report the presence and use of clues (low faithfulness scores on benchmarks), and explicitly treats “knowingly presenting hallucinated or incorrect information” as a concerning category to monitor. This is an important reminder: asking a model to “show its reasoning” does not automatically guarantee the reasoning is a faithful account of what caused the output.
The real fix: stop rewarding painted vases
No major platform claims hallucinations are eliminated, and public research argues that hallucinations are deeply tied to how systems are trained and, crucially, how they are scored and rewarded.
So the practical question becomes: what does a “reality‑based” AI practice reward?
It rewards traceability: grounded answers with citations to artefacts that can be inspected.
It rewards calibrated uncertainty: the ability to abstain, ask for clarification, or downgrade claims when evidence is missing, instead of filling gaps with fluent invention.
It rewards verification as “definition of done”: layered checks, monitoring, and incident response processes that treat hallucinations as reliability risks, consistent with risk‑management frameworks that emphasise governance, measurement, and continuous monitoring across the AI lifecycle.
And it resists the deeper cultural failure mode: reshaping work so that what is easy to generate, summarise, and score becomes what is treated as true. That is the emperor’s mistake, praising the painter not for matching the vase, but for making the demand look satisfied.
The operational lesson is simple and difficult: in any environment where truth is messy, costly, or uncertain, you must design incentives so the system, and the humans around it, prefer “the vase is real” over “the wall looks like it.”