An AI hallucination is an output that an AI system presents as plausible or factual but that is false, unsupported or inconsistent with its source material. In generative language models, fluent wording can make an invented detail look reliable even when the model has no dependable basis for it.

Language models generate text by estimating likely continuations from patterns in their training and context. They are not automatically checking every claim against a verified source. A request for a precise citation, a fact missing from the prompt or an unusual question can therefore produce confident but incorrect content. Retrieval tools, grounding and validation can reduce some errors, but each has limits and can fail in its own way.

Architecture and governance

Hallucination is a property of the architecture, not a failure of it. This does not mean every inaccurate output is harmless or that organisations should accept it without controls. It means that governance should plan for plausible errors as an expected system behaviour, rather than treating a model as a database that only needs occasional correction.

The level of checking should match the consequences of use. A brainstorming assistant may need a clear reminder that suggestions require review. A system used for medical, legal, financial or employment decisions needs stronger validation, authoritative sources, documented review and a clear limit on what it can decide. A human reviewer must have the information and time to check the answer, rather than simply approve a screen full of fluent text.

Reduce exposure and make errors visible

Organisations can test likely failure modes before deployment, including missing information, ambiguous prompts and adversarial inputs. They should measure factual accuracy on representative tasks, inspect citations and source links, and test whether the system signals uncertainty appropriately. Evaluation needs to continue after release because changes to prompts, models, retrieval sources and user behaviour can alter results.

Design can make verification easier: show source material, distinguish retrieved facts from generated wording and give users a straightforward way to flag errors. Policies should identify which outputs require independent confirmation and prohibit unsupported answers from being used as the sole basis for high-impact decisions. Incident reporting can reveal recurring patterns and inform updates to training, prompts, controls or intended use.

Users also need guidance that fluency is not evidence. If an output affects another person, money, safety or a formal record, responsibility remains with the people and organisation using the system. The relevant question is not whether a model can sometimes be wrong, but whether the system is designed so that errors can be found, contained and corrected before they cause avoidable harm.