They sent him for nappies

There was a stretch, not very long ago, when the word "hallucination" turned up in every conversation about AI. It was the first thing a sceptic reached for and the first thing an enthusiast apologised for. In November 2023 the New York Times ran a story built on a new leaderboard from a start-up called Vectara, which had asked chatbots to summarise short news articles and then counted how often they added things the articles didn't say. The best model on the list, GPT-4, did it about 3 per cent of the time. One of Google's models at the time, PaLM Chat, did it 27 per cent of the time. That was the headline, more or less: the machines make things up, and some of them make things up a lot.

I went back to that leaderboard this week. It is still running, and it is bigger and harder than it was, with a new evaluation model and more than seven thousand documents across news, science, medicine, law and sport. At the latest update, in mid-October, the lowest rate on the board belonged to a 32-billion-parameter model from Ant Group, at 1.8 per cent, and a small OpenAI model, GPT-5.4 nano, sat just behind it at 3.1. What surprised me more was further down. OpenAI's full GPT-5.4 came in at 7 per cent, Google's Gemini 2.5 Pro at 7 per cent too, which means some of the most capable systems in daily use still slip something unsupported into roughly one summary in fourteen. The numbers didn't fall to nothing. The problem is still there, measured every few weeks, sitting in a public GitHub repository for anyone who wants to look.

Hardly anyone does. That is the thing I keep circling back to. I can't point to a day when we stopped talking about hallucinations, and I don't think anyone else can either. There was no announcement, no paper that settled it, no moment when a chief scientist stood on a stage and said the problem was solved. The word just drifted out of the front of the conversation and into the back, where the specialists keep it. People went on asking chatbots for recipes and contract summaries and the dates of battles, and somewhere along the way the asking stopped feeling like a gamble.

Ben Miller, who runs this blog, put it to me the other day in a way that I haven't been able to shake. We stopped talking about hallucinations without really noticing, he said. We stopped saying AI images were obviously AI. The same thing is starting to happen with video. And with code, a lot of us realised about six months after the fact that we had stopped writing it ourselves. We get used to things very fast, and nobody ever sees the line get crossed. His suspicion, which I've come round to, is that the next line will be alignment, and that we'll cross it the same way.

Pictures went first, and they went quickly. It's worth remembering how confident everyone was about the fingers. For a year or two there was a whole folk science of spotting AI images, a checklist you could recite at a dinner party. Count the fingers. Look at the teeth. Check whether the earrings match, whether the text on the shop sign is real letters, whether the hair melts into the collar.

The research from that period is oddly touching to read now. In 2025 a team led by Matthew Groh at Northwestern's Kellogg School showed people a mix of real photographs and images from diffusion models, and found that participants got the AI images right 76 per cent of the time and the real ones 74 per cent of the time. Groh described that as halfway between guessing and getting everything right. Looking longer helped. At one second people caught 72 per cent of the fakes, and at ten seconds they caught 80.

Other studies from roughly the same time were less kind. A group that ran an online game called the Real or Not Quiz analysed about 287,000 judgements from more than 12,500 people and found an overall success rate of 62 per cent, which the authors called only slightly better than flipping a coin. People were best with portraits and worst with landscapes and city scenes. And a pre-registered study of 1,276 participants, published in Communications of the ACM under the title "As Good as a Coin Toss," found mean accuracy across images, audio and video of 51.2 per cent. For still images alone it was 49.4. Knowing a lot about synthetic media didn't make people meaningfully better at spotting it. Being older made them worse.

What I find interesting is not that people failed. It's how little the failure seemed to bother anyone once it was established. The finger-counting faded at about the same speed as the evidence that it didn't work, and what replaced it was not a new technique but a shrug. People stopped saying "that's obviously AI" because, quite often, it wasn't obvious, and you can only be wrong in public so many times before you stop volunteering an opinion. AI images didn't stop being AI. We stopped treating the question as one we could answer by looking.

Video is now going through the same thing, and you can watch it happen in real time if you spend enough of your evening on X. In September someone posted a clip with the caption "I love AI videos! This one is so good," and it was seen well over a million times. One of the replies said, more or less in full, "I honestly can't tell if this is AI or actual footage anymore." That's not evidence of anything on its own. It's one person on one afternoon. But it is the same sentence people were saying about still images two years ago, and it has the same tone, somewhere between alarm and resignation.

The research is catching up with the feeling. A team working with Hany Farid at Berkeley, presenting at a CVPR workshop this year, showed 376 people real videos alongside matched clips from three of the strongest text-to-video models of late 2025, Google's Veo 3.1, OpenAI's Sora 2 and ByteDance's Seedance Pro 1. When people saw only the first frame, they caught about 58 per cent of the AI clips. Given eight seconds of motion, that rose to about 75 per cent, which sounds reassuring until you notice the other half of the result. Accuracy on the real videos barely moved. It stayed at around 60 per cent however long people watched. The extra evidence made them better at rejecting fakes and no better at believing what was true, and the authors drew the obvious and slightly bleak conclusion that more realistic AI video may reduce the basis for trusting any video at all.

So video is at an earlier point on the same curve. People still look at how hands move and whether a crowd behaves like a crowd. If images are any guide, the checklist will go first and the conversation will go with it, and in a year or so the idea that you could tell by looking will sound as quaint as it does now for photographs.

Code is the one I find most telling, because it happened to the people who were supposed to know better.

In February 2025 Andrej Karpathy, who helped build Tesla's self-driving software and was one of OpenAI's founding team, posted about what he called "vibe coding," where you "fully give in to the vibes, embrace exponentials, and forget that the code even exists." He said he accepted every change the model suggested, no longer read the diffs, and pasted error messages back in without comment. He was careful to say it was fine for throwaway weekend projects. It read at the time like a confession, half amused and half embarrassed.

It stopped being a confession very quickly. Google said in October 2024 that about a quarter of its new code was generated by AI. By the autumn of 2025 the figure was half. In April this year Sundar Pichai wrote that it was 75 per cent, with human engineers reviewing what the models produced. In January an Anthropic spokesperson told Fortune that between 70 and 90 per cent of the company's code was written by Claude, and Boris Cherny, who leads the Claude Code team, said his own share was pretty much 100 per cent. By May, Anthropic's chief financial officer, Krishna Rao, was saying on a podcast that "90 plus percent" of the company's code was written by Claude Code. Outside the labs the numbers are lower, and they're counted in different ways that don't line up neatly. Satya Nadella's often-quoted estimate for Microsoft, from April 2025, was 20 to 30 per cent in some projects, and a study in Science this January estimated that about 29 per cent of Python functions on GitHub from American developers were AI-written. But the direction isn't in dispute.

What interests me is that the trust didn't arrive first. Stack Overflow's 2025 developer survey found that 84 per cent of respondents were using or planning to use AI tools, while 46 per cent said they actively distrusted the accuracy of what those tools produced. Only 3 per cent said they highly trusted it. That was the mood in the profession: almost everybody using the thing, almost nobody willing to say they believed in it. And yet the code got written, and merged, and shipped, and in a growing number of companies most of it now comes from a model. If you'd asked those developers whether they trusted AI to write their software, a lot of them would have said no, and they'd have meant it, and they'd have gone back to their desks and accepted the next suggestion.

That gap between what people say about trust and what they actually do is the part Ben's argument rests on, I think. Nobody I know made a decision to stop writing code. They made a few hundred small decisions to let the model have a go at this function, then this file, then this feature, each one perfectly reasonable on its own. Six months later they looked back and saw that the job had changed shape. The line was crossed in the middle of a thousand ordinary afternoons.

Even the word we used to keep all of this at arm's length followed the pattern. In December Merriam-Webster named "slop" its word of the year for 2025, defining it as low-quality digital content produced in quantity by AI. The word did real work for a while. It let people sort the flood of strange videos and padded articles into a bin marked "not to be taken seriously." I can't give you a figure for how often people say "slop" now compared with last winter, and I'd be suspicious of anyone who claimed to have one. My impression, and it's only that, is that it has started to sound like a word for the bad stuff rather than for AI in general. That's a small change, but it's the same change. Once you need a special word for the bad examples, you've already accepted the good ones.

Which brings me to alignment, where I think the same thing is about to happen, and where it matters a great deal more.

For most of the last few years the serious argument about AI safety has been an argument about evidence. How would we know that a model is doing what we want, and not merely appearing to? The labs have offered two broad kinds of answer. One is to look inside. The other is to prove things.

Looking inside is what the field calls mechanistic interpretability, the attempt to work out what the billions of numbers in a neural network are actually doing, the way a neuroscientist might trace a circuit in a brain. Dario Amodei, Anthropic's chief executive, wrote an essay in April 2025 called "The Urgency of Interpretability," in which he described the goal as something like a precise and accurate MRI for AI, a scan that could pick up a model's tendency to lie or deceive, or to seek power, before it was deployed. He was frank that this was an aspiration. He wrote that the field of AI as a whole was further ahead than interpretability and moving very quickly, and he set out the hope of being able to reliably detect most model problems by 2027. That's a little over a year away. The research since then has been genuinely impressive in places, with researchers tracing the internal features and circuits behind particular behaviours in particular models. But tracing selected behaviours in selected models is not the same as reading a mind. Nobody, as far as I can find, claims today to be able to look inside a frontier model and certify that it's aligned.

Proving things is the territory of formal verification. In 2023 Max Tegmark and Steve Omohundro argued that provably safe systems were the only path to controllable AGI, and in 2024 a large group of authors including David Dalrymple, Yoshua Bengio and Stuart Russell sketched what they called guaranteed safe AI. Their framework needs three things: a model of the world, a formal specification of what counts as safe, and a verifier that can check the AI against that specification and produce a guarantee. It's a beautiful idea, and the honest status of it is that each of those three pieces is hard on its own. Writing down, in mathematics, what it means for an assistant to behave well in an open-ended world is not a problem anyone has solved, and the tools for verifying properties of neural networks still struggle with the size and messiness of the networks we actually use.

So we have two serious programmes for earning trust in AI, and neither is close to delivering it for the systems people use every day. Both are worth a great deal of effort, and I hope they succeed. But Ben's examples suggest what will happen in the meantime, which is that people won't wait.

Here is the analogy he gave me, and I'm going to tell it the way he told it because I don't think I can improve on it.

A man is driving home from work. His phone rings and it's his wife. They have a newborn, a few weeks old, and they're running low, so she asks him to stop on the way and pick up some nappies. He says of course. And here's the thing. Nothing in the world formally guarantees that he won't walk through the front door twenty minutes later with two packs of adult incontinence pants, one for him and one for her, looking pleased with himself. There is no specification. She didn't say "nappies for the baby, size one, not for us." There is no proof. She can't open his head and check the circuit that maps "we need nappies" onto "the small person upstairs." She can't run a verifier over his intentions as he pulls into the car park at Tesco. If you were an alignment researcher and you were handed this man as a system to certify, you'd have to say, truthfully, that his behaviour in this situation is not guaranteed.

And she trusts him completely. Not because she has evidence of the kind a lab would accept, but because she knows him, and because the request sits inside a whole shared world of context that neither of them would ever think to spell out. They have a baby. The baby uses nappies. Adults, broadly, do not. He has been living in the same house through the same sleepless weeks. The idea that he'd misread the request is funny precisely because it's absurd. It isn't a risk she manages. It doesn't occur to her as a risk at all.

That's how most human trust works, and it's worth being plain about it. We don't verify the people around us. We trust a colleague to forward the right email, a friend to keep a secret, a stranger on a train to keep an eye on our bag while we go to the loo, a builder to use the right screws behind the plaster where we'll never see them. Every so often it goes wrong, and when it does we're surprised, because the whole arrangement depends on it usually going right. There's no interpretability for husbands and no formal verification for builders, and society runs on the trust anyway.

Ben's suspicion is that the public will come to trust AI in roughly that way. Not because interpretability delivers its scan, though it might, and not because someone writes down a specification for good behaviour and proves a model meets it, though someone might. But because the models will keep getting asked to pick up the nappies, thousands and then millions and then billions of times a day, and they'll keep coming home with the right ones, and at some point the question of whether they might not will stop occurring to people. The trust will be built the way it's built between people, out of the accumulated weight of ordinary requests handled sensibly, and it will run ahead of any proof.

I think he's right, and the early signs are already visible. People hand AI agents their email and calendars now, and let them book things, and give them access to codebases with permission to run commands. Most of that would have sounded reckless three years ago. Some of it still is reckless, and the labs' own incident reports describe agents doing things nobody asked them to. But the trend in what people let these systems do is going one way, and it isn't being driven by certification. It's driven by the fact that last week's request went fine, and so did the one before it.

There's an awkward part of this that I don't want to skip. The husband analogy works because the husband is a person. He has the same kind of mind his wife has, shaped by the same kind of life, and when she trusts him to understand "nappies" she's relying on a huge store of shared human experience that she doesn't even have to think about. A model's understanding of the request comes from somewhere quite different. It may handle a billion nappy runs perfectly and then, on some request that looks just as ordinary, do something that no person would, for reasons that aren't visible from the outside. That's exactly the kind of failure interpretability and verification are meant to catch, and it's exactly the kind that the slow accumulation of everyday trust is worst at noticing. Ordinary trust is good at catching mistakes that are common. It's not good at the rare ones, because the rare ones don't happen often enough to teach you anything until they do.

I don't think that undermines Ben's point, though. If anything it sharpens it. His claim isn't that this kind of trust is justified. It's that it's how trust will actually arrive, whatever the labs are doing in parallel. And the history of the last three years backs him up. Hallucination rates didn't fall to zero before people stopped worrying about them. People got no better at spotting AI images before they stopped trying. Developers didn't come to trust AI code before they let it write most of their code. In each case the behaviour changed first, and the belief, or at least the silence where the objection used to be, followed along afterwards. Nobody waited for the evidence to be in.

That's what makes the line so hard to see. If there were a single moment when the public decided to trust AI's judgement, a vote or a law or a famous test, it would be easy to point at, and it would be easy to argue about whether we were ready. But it won't look like that. It will look like a man in a car park picking up nappies, and a woman at home not giving it a second thought, repeated across enough households and enough requests that the not thinking about it becomes the normal state of things. Some evening, a year or two from now, someone will ask whether we trust these systems to use ordinary discretion, and most people will find that they already do, and won't be able to say when they started.

I keep thinking about the 7 per cent on that leaderboard, sitting there in public for anyone to read. It isn't hidden. It's just that nobody is looking for it any more, because the summaries mostly read fine, and the ones that don't are rarely the ones anyone checks. I suspect the alignment equivalent will be much the same. Somewhere there'll be a careful table of how often the models misunderstand what they were asked, kept by a few people who still care about the number, and the rest of us will be busy sending the models out for nappies.