OpenAI has said that an AI model will never be able to generate fully accurate responses regardless of the size of the model or its search and reasoning capabilities.
AI models will never reach 100 per cent accuracy because some real-world questions are “inherently unanswerable,” the ChatGPT-maker said in a blog post about its latest research paper published on Friday, September 5.
The new research is focused on why large language models (LLMs) like GPT-5 continue to hallucinate despite advances in AI research and development. Notably, it seems to make no mention of artificial general intelligence (AGI) or artificial superintelligence (ASI), which currently remain theoretical but are often described as key next stages in the evolution of current multi-modal AI systems.
OpenAI researchers have found that hallucinations in LLMs are rooted in standard training, particularly in the pre-training phase where AI models are trained on vast troves of text or data to predict the next token. Another key reason for hallucinations in LLMs is evaluation procedures that often reward AI-generated responses that involve guesswork and penalise AI models for openly acknowledging that they do not have an answer.
As a result, small language models like GPT-4o-mini are said to generate fewer hallucinated responses as they know their limits. “For example, when asked to answer a Māori question, a small model which knows no Māori can simply say “I don’t know” whereas a model that knows some Māori has to determine its confidence,” OpenAI said.
The Microsoft-backed AI startup’s new research on AI-generated hallucinations comes nearly a month after the bumpy debut of its latest flagship AI model, GPT-5. “GPT‑5 has significantly fewer hallucinations especially when reasoning, but they still occur,” it said.
What are hallucinations?
There are several definitions for ‘information hallucination’, especially in the context of generative AI. However, OpenAI defines the term as “instances where a model confidently generates an answer that isn’t true.” “ Hallucinations are plausible but false statements generated by language models,” it added, citing an example of a ‘widely used chatbot’ that confidently gave wrong answers when asked for the title of the research paper or the birthday of one of its authors.
Story continues below this ad
According to OpenAI, hallucinations have continued to persist because current evaluation methods of AI models set the wrong incentives. “Most evaluations measure model performance in a way that encourages guessing rather than honesty about uncertainty,” it said.
The patterns in training data fed to an AI model also play a key role in how often it hallucinates. For instance, spelling errors fade because the pattern becomes more consistent as models scale. However, rare facts such as a pet’s birthday that do not appear as frequently in training datasets makes AI models more likely to cause hallucinations.
How to address hallucinations?
In its research paper, OpenAI called for existing evaluation methods to be modified in order to penalise confident AI-generated errors more than uncertain AI-generated responses during the post-training/reinforcement learning stages. This would allow the model to be fine-tuned to acknowledge uncertainty instead of confidently giving wrong answers.
While some evaluation tests in the past have sought to discourage blind guessing through negative marking or partial credit for blank questions, OpenAI said that these processes do not go far enough.
Story continues below this ad
Instead, the company has called for all accuracy-based benchmarking tests to be updated in order to discourage guessing. “If the main scoreboards keep rewarding lucky guesses, models will keep learning to guess. Fixing scoreboards can broaden adoption of hallucination-reduction techniques, both newly developed and those from prior research,” OpenAI said.
