Can people be persuaded not to believe disinformation?:

Dr [Thomas] Costello believes chatbots work where humans fail because they offer rational responses instead of letting emotions get the better of them. What’s more, they are able to comb through their extensive training data to offer precise counter-arguments, rather than the generalised ones humans often reach for in debates.

Researchers lift the lid on how reasoning models actually “think”:

When Claude itself is asked to reason, printing out the chain of thought that it takes to answer maths questions, the microscope suggests that the way the model says it reached a conclusion, and what it actually thought, might not always be the same thing. Ask the llm a complex maths question that it does not know how to solve and it will “bullshit” its way to an answer: rather than actually trying, it decides to spit out random numbers and move on.
Worse still, ask a leading question — suggesting, for instance, that the answer “might be 4” — and the model still secretly bullshits as part of its answer, but rather than randomly picking numbers, it will specifically insert numbers that ultimately lead it to agree with the question, even if the suggestion is wrong. 

These stories were posted just a few days apart. It’s comical to me how many AI researchers act as though the hallucinations and bullshitting simply don’t exist. Also: LLMs are not rational or irrational or emotional or anything else that human beings are. They are the conduits, thanks to their corpora, of human rationality or irrationality or emotionalism. 

The discourse on “AI” systems, chat bots, “assistants” and “research helpers” is defined by a lot of future promises. Those systems are disfunctional or at least not working great right now but there’s the promise of things getting better in the future.

Which is how we often perceive tech to work: Early versions might be a bit wonky, but there’s constant iteration and work going on to improve systems to be more capable, more robust and maybe even cheaper at some point.

The most pressing problem for many modern “AI” systems, especially the generative systems that are all the rage these days are so-called “hallucinations” which is a term describing when an AI system generates incorrect information. Think a research agent inventing a paper to quote from that doesn’t exist for example (Google’s AI assistant telling you to put glue on pizza is not a hallucination in that regard because that is just regurgitating information from Reddit that every toddler would recognize as a joke). Hallucinations are the big issue that many researchers are trying to address – which mixed results. Methods like RAG are shifting the probabilities a bit but are still not solving the problem: Hallucinations keep happening.

But I think that this discourse misses an important thing: Anything an LLM generates is a hallucination.

That doesn’t mean that everything LLMs generate is incorrect, far from it. What I am referencing is what hallucinations are actually defined as: A hallucination is a perception you have that is not connected to any actual stimulus. You hallucinate when you perceive something in the world that you have no sensor data for.

The term hallucination itself is an anthropomorphization of those statistical systems. They don’t “know”, or “think” or “lie” or do any such things. They iteratively calculate the most probable set of words and characters based on the original data. But if we look at how it is applied to “AI”s I think there is a big misunderstanding because it creates a difference between true and false statements that just isn’t there.

For humans we separate “real perceptions” from hallucinations by the link to sensor data/stimulants: If there is an actual stimulant of you feeling a touch it’s real, if you just think you are being touched, it’s a hallucination. But for LLMs that distinction is meaningless.

A line of text that is true has – for the LLM – absolutely no different quality than one that is false. There is no link to reality, no sensor data or anchoring, there’s just the data one was trained on (that also doesn’t necessarily have any connection to reality). If using the term hallucination is useful to describe LLM output it is to illustrate the quality of all output. Everything an LLM generates is a hallucination, some just might accidentally be true.

And in that understanding the terminology might actually be enlightening, might actually help people understand what those systems are doing and where it might be appropriate to use and – more importantly – where not.

Who wouldn’t like a computer you can just talk to? LLMs are good enough at this to be impressive! You can ask it stuff naturally and get back a reasonable-looking response! This is genuinely interesting and it’s a genuine advance!

But “impressive” doesn’t mean “good enough for production use.” When you try to use an LLM as a search engine, they’re famous for getting things stupidly wrong and making stuff up. You don’t really want wood glue on your pizza and you shouldn’t eat a small rock every day. [BBC, 2024]

Columbia Journalism Review ran some chatbots through their paces. CJR picked random news articles from various publishers, gave the bots a text excerpt, and asked for the article’s headline, publisher, publication date, and web address. [CJR]

Across all bots, over 60% of answers were wrong. Perplexity was least-worst at 37% wrong, and Grok was 94% wrong.

All the bots were very confident in their wrong answers. Paid bots were more confidently incorrect than the free ones.

Licensing deals don’t seem to have made OpenAI any better at finding the content they’ve done a deal to use.

The search bots also ignored the robots.txt file – which tells search engines “please don’t scan here.” Perplexity Pro was the worst for this.

CJR did a similar test in November last year. The bots haven’t improved. [CJR, 2024]

But AI’s been full of impressive demos that didn’t work as products for the past 70 years. Stop shoving chatbots into places they just can’t do the job.

• Video version

The Constitution gives to Congress, not the president, the power to impose tariffs. But the International Emergency Economic Powers Act allows the president to impose tariffs if he declares a national emergency under the National Emergencies Act, which Trump did on February 1. That same law allows Congress to end such a declaration of emergency, but if such a termination is introduced — as Democrats have recently done — it has to be taken up in a matter of days.

But this would force Republicans to go on record as either supporting or opposing the unpopular economic ideology Trump and Musk are imposing. So Republicans just passed a measure saying that for the rest of this congressional session, “each day ... shall not constitute a calendar day” for the purposes of terminating Trump’s emergency declaration.

The Republicans’ legislation that a day is not a day seems to prove the truth of [Edmund] Burke’s observation that by trying to force reality to fit their ideology, radical ideologues will end up imposing tyranny in the name of liberty.