AI hallucination
An AI hallucination is a statement from an AI model that sounds plausible but is false, such as a made-up measurement, feature or guarantee.
- Hallucination
What it means
Google’s Machine Learning Glossary describes a hallucination as “plausible-seeming but factually incorrect output by a generative AI model”.1 The model does not flag it as a guess. It can read just like the true sentences around it.
It happens because a language model writes what is likely, not what it has checked. Asked to describe a product with gaps in its data, it may fill them with details typical of similar products rather than true of this one.
In product copy the risk is a figure or feature that reads as fact: a capacity, a guarantee, a compatible model. Google’s rule for product details suits all generated copy: “Only provide an attribute name and value when the value is confirmed”.2
Think of it like this
It is like a confident stranger giving directions to a street they have never visited. The directions sound sure, use real street names, and still send you the wrong way.
For example
A bath’s row holds its type, size, colour and capacity, but no weight. Asked for a full description, a model writes “weighs just 28 kg”. Nothing in the row says so: the figure is a hallucination, however plausible.
Why it matters
An invented detail in your copy becomes an invented detail in an assistant’s answer, and can mean a return when the product arrives. UK law counts “the provision of false or misleading information relating to a product” as a misleading action.3 That is unfair if it is likely to lead the average consumer to a decision they would not otherwise have taken.4
Not to be confused with
Filling a gap is not always a hallucination. Product data enrichment fills a missing value from a source you can point to; a hallucination fills it from nowhere.
Sources
- Google for Developers. Machine Learning Glossary.
- Google Merchant Center Help. Product detail [product_detail].
- legislation.gov.uk. Digital Markets, Competition and Consumers Act 2024, section 226 (Misleading actions).
- legislation.gov.uk. Digital Markets, Competition and Consumers Act 2024, section 225 (Prohibition of unfair commercial practices).