The discourse on “AI” systems, chat bots, “assistants” and “research helpers” is defined by a lot of future promises. Those systems are disfunctional or at least not working great right now but there’s the promise of things getting better in the future.

Which is how we often perceive tech to work: Early versions might be a bit wonky, but there’s constant iteration and work going on to improve systems to be more capable, more robust and maybe even cheaper at some point.

The most pressing problem for many modern “AI” systems, especially the generative systems that are all the rage these days are so-called “hallucinations” which is a term describing when an AI system generates incorrect information. Think a research agent inventing a paper to quote from that doesn’t exist for example (Google’s AI assistant telling you to put glue on pizza is not a hallucination in that regard because that is just regurgitating information from Reddit that every toddler would recognize as a joke). Hallucinations are the big issue that many researchers are trying to address – which mixed results. Methods like RAG are shifting the probabilities a bit but are still not solving the problem: Hallucinations keep happening.

But I think that this discourse misses an important thing: Anything an LLM generates is a hallucination.

That doesn’t mean that everything LLMs generate is incorrect, far from it. What I am referencing is what hallucinations are actually defined as: A hallucination is a perception you have that is not connected to any actual stimulus. You hallucinate when you perceive something in the world that you have no sensor data for.

The term hallucination itself is an anthropomorphization of those statistical systems. They don’t “know”, or “think” or “lie” or do any such things. They iteratively calculate the most probable set of words and characters based on the original data. But if we look at how it is applied to “AI”s I think there is a big misunderstanding because it creates a difference between true and false statements that just isn’t there.

For humans we separate “real perceptions” from hallucinations by the link to sensor data/stimulants: If there is an actual stimulant of you feeling a touch it’s real, if you just think you are being touched, it’s a hallucination. But for LLMs that distinction is meaningless.

A line of text that is true has – for the LLM – absolutely no different quality than one that is false. There is no link to reality, no sensor data or anchoring, there’s just the data one was trained on (that also doesn’t necessarily have any connection to reality). If using the term hallucination is useful to describe LLM output it is to illustrate the quality of all output. Everything an LLM generates is a hallucination, some just might accidentally be true.

And in that understanding the terminology might actually be enlightening, might actually help people understand what those systems are doing and where it might be appropriate to use and – more importantly – where not.

Who wouldn’t like a computer you can just talk to? LLMs are good enough at this to be impressive! You can ask it stuff naturally and get back a reasonable-looking response! This is genuinely interesting and it’s a genuine advance!

But “impressive” doesn’t mean “good enough for production use.” When you try to use an LLM as a search engine, they’re famous for getting things stupidly wrong and making stuff up. You don’t really want wood glue on your pizza and you shouldn’t eat a small rock every day. [BBC, 2024]

Columbia Journalism Review ran some chatbots through their paces. CJR picked random news articles from various publishers, gave the bots a text excerpt, and asked for the article’s headline, publisher, publication date, and web address. [CJR]

Across all bots, over 60% of answers were wrong. Perplexity was least-worst at 37% wrong, and Grok was 94% wrong.

All the bots were very confident in their wrong answers. Paid bots were more confidently incorrect than the free ones.

Licensing deals don’t seem to have made OpenAI any better at finding the content they’ve done a deal to use.

The search bots also ignored the robots.txt file – which tells search engines “please don’t scan here.” Perplexity Pro was the worst for this.

CJR did a similar test in November last year. The bots haven’t improved. [CJR, 2024]

But AI’s been full of impressive demos that didn’t work as products for the past 70 years. Stop shoving chatbots into places they just can’t do the job.

• Video version

The Constitution gives to Congress, not the president, the power to impose tariffs. But the International Emergency Economic Powers Act allows the president to impose tariffs if he declares a national emergency under the National Emergencies Act, which Trump did on February 1. That same law allows Congress to end such a declaration of emergency, but if such a termination is introduced — as Democrats have recently done — it has to be taken up in a matter of days.

But this would force Republicans to go on record as either supporting or opposing the unpopular economic ideology Trump and Musk are imposing. So Republicans just passed a measure saying that for the rest of this congressional session, “each day ... shall not constitute a calendar day” for the purposes of terminating Trump’s emergency declaration.

The Republicans’ legislation that a day is not a day seems to prove the truth of [Edmund] Burke’s observation that by trying to force reality to fit their ideology, radical ideologues will end up imposing tyranny in the name of liberty.