AI's "making stuff up" problem won't go away

AI makers could do more to limit chatbots' penchant for "hallucinating," or making stuff up — but they're prioritizing speed and scale instead.Why it matters: High-profile AI-induced gaffes keep embarrassing users and the technology's unreliability continues to cloud its adoption.The big picture: Hallucinations aren't quirks — they're a foundational feature of generative AI that some researchers say will never be fully fixed.AI models predict the next word based on patterns in their training data and the prompts a user provides. They're built to try to satisfy users, and if they don't "know" the answer they guess.Chatbots' fabrications were a problem when OpenAI CEO Sam Altman introduced ChatGPT in 2022, and the industry continues to remind users that they can't trust every fact a chatbot asserts.Every week brings painful new evidence that users are not listening to these warnings.Last week it was a report from Robert F. Kennedy Jr.'s Health and Human Services department citing studies that didn't exist. Experts found evidence suggesting OpenAI's tools were involved.A week earlier, The Chicago Sun-Times published a print supplement with a summer reading list full of real authors, but hallucinated book titles.AI legal expert Damien Charlotin tracks legal decisions in which lawyers have used evidence that featured AI hallucinations. His database details 30 instances in May 2025. Legal observers fear the total number is likely much higher.Yes, but: AI makers are locked in fierce competition to top benchmarks, capture users and wow the media. They'd love to tamp down hallucinations, but not at the expense of speed.Chatbots could note how confident the language model is in the accuracy of the response, but they have no real incentive for that, Tim Sanders, executive fellow at Harvard Business School and VP at software marketplace G2, told Axios. "That's the dirty little secret. Accuracy costs money. Being helpful drives adoption."Between the lines: AI companies are making efforts to reduce hallucinations, mainly by trying to fill in gaps in training data.Retrieval augmentation generation (RAG) is one process for grounding answers in contextually relevant documents or data.RAG connects the model to trusted data so it can retrieve relevant information before generating a response, producing more accurate answers.AWS offers Amazon Bedrock, a cloud service that allows customers to use various AI providers and responsible AI capabilities (including reducing hallucinations) to build generative AI applications.AWS says its Bedrock Guardrails can filter over 75% of hallucinated responses.Researchers from Google's DeepMind, along with Stanford University and University of Illinois at Urbana-Champaign, are working on a Search-Augmented Factuality Evaluator (SAFE), which uses AI to fact check AI.Anthropic offers a guide to help developers limit hallucinations — including allowing the model to answer, "I don't know."OpenAI's developer guide includes a section on how much accuracy is "good enough" for production, from both business and technical perspectives.Researchers inside AI companies have raised alarms about hallucinations, but those warnings aren't always front of mind in organizations hell-bent on a quest for "superintelligence."Raising money for building the next big model means companies have to keep making bigger promises about chatbots replacing search engines and AI agents replacing workers. Focusing on the technology's unreliability only undercuts those efforts.The other side: Some AI researchers insist that the hallucination problem is overblown or at least misunderstood, and that it shouldn't discourage speedy adoption of AI to boost productivity."We should be using [genAI] twice as much as we're using it right now," Sanders told Axios. Sanders disputes a recent New York Times article suggesting that as AI models get smarter, their hallucination rates get worse. OpenAI's o3 and similar reasoning models are designed to solve more complex problems than regular chatbots. "It reasons. It iterates. It guesses over and over until it gets a satisfactory answer based on the prompt or goal," Sanders told Axios. "It's going to hallucinate more because, frankly, it's going to take more swings at the plate." Hallucinations would be less of a concern, Sanders says, if users understood that genAI is designed to make predictions, not verify facts. He urges users to take a "trust, but verify" approach.The bottom line: Hallucination-free AI may be a forever problem, but for now users can choose the best model for the job and keep a human somewhere in the loop.

Comments

World news

Independent

Elon Musk’s business empire was built on government help. How badly could Donald Trump hurt him?

about 2 hours ago

Independent

Paramount chair Shari Redstone confirms cancer diagnosis amid high-stakes sale linked to Trump’s $25 million ‘60 Minutes’ lawsuit

about 2 hours ago

Paul Hollywood: ‘If you want a photo just ask – don’t secretly film me’

The i Paper

Comments

Similar News

AI is remaking everything

English-speaking countries more nervous about rise of AI, polls suggest

Plans for 'AI Fight Club' of defense, tech companies brewing in Washington

Reddit sues AI company Anthropic for allegedly ‘scraping’ user comments to train chatbot

DreamWorks co-founder Katzenberg likens AI to CGI revolution

Weather Company CEO on AI and what's next for forecasting

Viral video of Chinese paraglider likely includes faked AI footage

Super Trouper meets supercomputer: AI helping Abba star to write musical

Top moments from Axios' AI+ Summit in New York

Ministers offer concessions on AI and copyright to avoid fifth Lords defeat

Behind the Curtain: Your AI survival kit

Google engineer says humans can cheat death – we just have to merge with AI

Meta signs deal with nuclear plant to power AI and datacenters for 20 years

From drones and AI to nukes - terrifying ways UK's enemies could attack us

Youshua Bengio launches $30M lab to rethink AI design

U.S. restrictions on AI chips, student visas ‘violate’ tariffs truce: China

Facebook and Instagram owner Meta to enable AI ad creation by end of next year

My students think it’s fine to cheat with AI. Maybe they’re onto something.

Behind the Curtain: Trump risks pushing world into China's arms on AI

AI is remaking — and breaking — the web

World news

Elon Musk’s business empire was built on government help. How badly could Donald Trump hurt him?

Paramount chair Shari Redstone confirms cancer diagnosis amid high-stakes sale linked to Trump’s $25 million ‘60 Minutes’ lawsuit

Paul Hollywood: ‘If you want a photo just ask – don’t secretly film me’

Bouncy castle operator cleared in tragedy that killed six

South Koreans Have a New President, and Mixed Emotions

U.K. Faces ‘Extraordinary’ Threat from Russian and Iranian Plots, Official Warns

Russia launches ballistic missile, drone attack on Kyiv after Ukraine's hit on bombers

Ukraine-Russia war latest: Trump warns Putin’s revenge for drone attack ‘not going to be pretty’

Ryan Reynolds acquires a new sports team — this time with Hugh Jackman

Trump Tower is throwing a Pride month party for LGBTQ+ Republicans despite White House dodging events

Kyiv under major drone attack after Putin threatened response to Ukrainian strikes in Russia

NATO is on the cusp of accepting Trump's 5% defense investment demand, Rutte says

Human rights committee blames Guatemala for forcing girl who was raped to carry out her pregnancy

Moon lander 'likely to have crashed', says Japan's ispace

The four words a top German detective said before 'last-ditch' Madeleine McCann search

Metro Vancouver’s top executive’s salary now tops half a million dollars

Fraser Highway revitalization creating headaches for Langley businesses

BC Conservatives question $1M contract for B.C. drug and mental illness adviser

Russia Launches Broad Assault on Kyiv and Other Cities in Ukraine

Navy launches frantic search for female sailor who vanished near her barracks a week ago

Republican senators question cost of Army’s parade spectacle set for Trump’s birthday

Oh the huge-manatee! Giant sculpture of marine creature goes missing in Florida

Mahmoud Khalil details psychological pain and reputational harm in new legal filings: ‘Efforts to erase my humanity’

Scottish Labour beat SNP and Reform UK in crucial Hamilton by-election

Suspect in Boulder Molotov attack facing 118 charges

Scoop: Musk claims spur Democrats to probe whether Trump is in the Epstein files

Brit boy, 2, dies after his family find him floating in the pool at Ibiza holiday home

RCMP take steps to combat thieves targeting Alberta’s evacuated communities

Nisga’a-owned LNG pipeline wins key approval as BC Greens, other First Nations push back

Winnipeg Sea Bears sign former Toronto Raptors guard

Here’s how to get a model-approved slick back bun

‘Unspeakable joy’: Lost B.C. cat reunied with owner after six years

Marc Garneau, First Canadian in Space, Dies at 76

Calgary senior stuck at home due to broken elevator, misses death of loved one

Clash of the titans: Immovable object of Trump ego met unstoppable force of Musk arrogance in trolling for the ages

Postal service van runs over and kills 84-year-old man who went outside to check his mail

Family of teenage girl allegedly tortured and murdered by man she met on Grindr sues app

Musk declares war on Trump as split over ‘big, beautiful’ bill gets personal

Democrats can barely contain their glee over Trump and Musk’s ‘big beautiful breakup’

Trump-Elon Musk feud live updates: Tesla CEO endorses push for president’s impeachment amid epic meltdown

Deadly Aid Chaos in Gaza

Trump’s Travel Ban Returns

Trump Resurrects Controversial Travel Ban With New Restrictions

Attempt to land private moon lander in jeopardy &#8211; as contact lost

Serving coffee in America, under fear of death from the Taliban in Afghanistan

VPD say nearly $150K lost in 70 distraction thefts this year alone

Banned knives, stolen Winners clothes, illegal tobacco seized from Edmonton shop

They’re back: Windstorm-wrecked vessels in Vancouver renew derelict boat concerns

First blast of summer heat headed for southern B.C.

Sask. woman last seen in Edmonton missing under suspicious circumstances: EPS

‘No answers and no supports’: Saskatchewan wildfire evacuees wait for aid

‘Just miss everything about him’: Cochrane lacrosse player dies in motocross accident

House GOP subpoenas Biden's personal physician

"Vendetta against Harvard": College challenges Trump's entry ban on new foreign students

"People don't even like him": Musk becomes GOP pariah after "ridiculous" Trump attacks

"Such ingratitude": Musk was the top Republican donor in 2024

India leads in remittances - but Trump's tax could deal a blow

Attempt to land private moon lander in jeopardy – as contact lost