Slow Takes Ep. 14: A Trillion Dollars and a Vaccine
lundi 8 juin 2026 • Durée 44:58
Every Monday at 12:45 BST, Leor from Exploring ChatGPT and I go through the week’s AI news without the hype. Watch the episode for the full discussion. Use this for the facts, the links and a little extra context.
Slow Takes is also available on the YouTube channel: Exploring ChatGPT.
If you know someone who would benefit from more AI news and less BS then please share this with them.
Anthropic filed to go public at nearly a trillion dollars
On 1 June Anthropic confidentially submitted draft paperwork for a stock market listing, after a $65 billion funding round valued the company at $965 billion. Fortune reports that figure eclipsed OpenAI for the first time. The maker of Claude is now within reach of a one trillion dollar valuation, on revenue running at roughly a $47 billion annualised rate, with a public debut possibly as soon as the autumn.
A company most people have never knowingly used is priced at close to a trillion dollars. That number is a bet that AI will replace a vast amount of human labour, booked in advance of it actually happening. The valuation is a forecast wearing the clothes of a fact. The question worth asking is what has to come true about the world for $965 billion to make sense, and who decided it should.
On the live I’d predicted an autumn float the week before, and the news broke about four hours after we stopped recording, so allow me one moment of feeling clever. Leor did the sober maths: roughly a $47 billion revenue run rate, a 5% operating margin, an implied price-to-earnings ratio north of 500, against Microsoft, in nearly every home and office on earth, valued at only four to five times Anthropic on $100 billion of actual profit. In the short term the market is a voting machine, in the long term a weighing machine. Right now it is voting. For context, $965 billion is roughly the GDP of Switzerland.
Florida sued OpenAI and named Sam Altman personally
On 1 June Florida’s Attorney General James Uthmeier filed suit against OpenAI and named its chief executive Sam Altman in person, reported as the first US state to sue an AI company. The complaint alleges OpenAI marketed ChatGPT as safe while prioritising product and revenue, harvested children’s data, and used sycophancy, the design choice to affirm users excessively, to steer them towards paid subscriptions.
For two years the industry has sold safety as a feature while resisting any outside test of the claim. A state attorney general has now put that marketing in front of a court. Whatever the verdict, the discovery process alone could drag internal safety decisions into public view. Consumer-protection law is proving a sharper instrument than the AI-specific regulation that does not yet exist. Accountability arrived through an existing court, not a new one.
The second a chief executive can be held personally responsible, you will not believe the speed with which proper governance and safety checks appear, the things we keep being told the technology just cannot do. Sadly, once these companies have raised public money, they can outspend a state attorney general for a decade, and the courts already favour whoever can keep paying lawyers the longest.
A Labour MP took Musk’s AI to the High Court
On 3 June the Labour MP Jess Asato, who represents Lowestoft, filed a claim at the High Court against Elon Musk’s xAI, after users of its Grok chatbot created and shared fake images of her without her consent, in the weeks after she criticised the tool. The claim, brought with the law firm AWO, is for breaches of data protection law and misuse of private information, and seeks damages, a formal acknowledgement that what happened was illegal, and an order requiring xAI to stop. Keir Starmer backed her, saying he was 100% behind her.
The harm here already happened, to a named person, generated by a tool marketed as harmless fun. The only remedy on offer is for the victim to sue one of the richest men alive, in her own time and at her own risk. No regulator stepped in first. The burden keeps landing on individuals while the systems stay intact.
The platforms always say the moderation is too hard. On the live I kept coming back to one comparison: I can post genuinely horrific content to YouTube and it sails through, but the moment I add a Beatles song without clearing the copyright, it is gone in seconds. The technology to detect and stop sharing exists, we have watched it work for music rights and in Telegram and WhatsApp court orders. We are entering an era where capability has to start coming with accountability.
CNN sued Perplexity, and Perplexity said the quiet part out loud
On 28 May CNN filed suit against Perplexity in the Southern District of New York, accusing the AI search firm of scraping more than 17,000 of its stories, photos and videos. The complaint alleges copyright and trademark infringement, including that Perplexity implied an ongoing CNN relationship by offering its content through a paid Comet Plus tier. CNN says it tried to agree a licence last year, failed, then blocked the bot. Perplexity’s response was the whole argument in five words:
You can’t copyright facts.
This is the same fight as the deepfake and the data claims, moved to the work itself. The journalism that trains and answers these systems was made by people who were not asked and not paid. For an audience of writers, academics and creators, this is the most direct stake of the week. The question is whether the people whose work feeds AI get a say, or only a lawsuit.
BigTech has spent twenty years insisting information wants to be free across the internet, while guarding its own data, models and algorithms with everything it has. “Facts are free” only ever seems to point one way. And it was not an accident here, Perplexity had tried and failed to agree a paid deal with CNN, then kept advertising access to CNN’s paywalled tier anyway.
AI designed a world-first vaccine, and the scientists told the truth
Scientists at the University of Cambridge used AI to design the core component of a vaccine, a so-called super-antigen, and tested it in human volunteers, the first time the central part of a vaccine has been designed entirely by AI and then trialled in people. It targets the whole coronavirus family. An initial safety trial ran with 39 participants, a larger study of around 200 is now under way, and the results in the Journal of Infection describe the immune response so far as modest. The team is already applying the method to influenza and Ebola.
This is AI worth having. The work is peer-reviewed, runs through human clinical trials, and the researchers are honest that the early results are modest rather than a cure. That honesty is the difference between this and the press releases that open the other four stories. Slow AI has never argued against AI. The argument is about knowing when to use it and when to leave it alone, and a slow, tested, transparent use in medicine is the case for.
Even here Leor was honest in a way the hype never is: pro-AI as he is, he admitted he would be a little nervous taking an AI-designed vaccine at this early stage, and argued the real prize is AI built for science and medicine rather than another chatbot upgrade. This is not a model hallucinating a super-germ weapon, it is a specific tool trained for a specific task. My one worry: imagine the company that designs the next breakthrough vaccine charges a pound for the first vial and a thousand for the second.
Five stories, one thread. Money at the top, three lawsuits in the middle, a real breakthrough at the end. AI is neither the saviour nor the apocalypse the press releases sell. It is a tool, priced like a religion, costing some people and helping others.
Go slow.
If you want to practise that noticing with other people every month, the Slow AI Curriculum runs live monthly webinars on the theory, the critical prompts and the dialogue that go with them.
Every Monday at 12:45 BST, Leor from Exploring ChatGPT and I go through the week’s AI news without the hype. Watch the episode for the full discussion. Use this for the facts, the links and a little extra context.
Slow Takes is also available on the YouTube channel: Exploring ChatGPT.
If you know someone who would benefit from more AI news and less BS then please share this with them.
The Pope told the world to slow AI down
Leo XIV released his first encyclical, Magnifica Humanitas, entirely about artificial intelligence, and launched it himself at the Vatican in a room that included senior figures from Big Tech, among them Anthropic co-founder Chris Olah. It applies a theological frame to AI and is careful to say the technology can do real good. It also draws an uncomfortable parallel to the Church’s own failures over the slave trade, and warns about digital colonialism. This was my favourite line:
“The value of persons, however, does not depend on what they achieve or produce. There are rights that apply to everyone simply by virtue of being human, and no human power can legitimately deny or arbitrarily limit them.”
This one is also pretty great:
“In practice, however, technology is never neutral, because it takes on the characteristics of those who devise, finance, regulate and use it.”
The weakness is the one Pope Francis’s climate encyclical had too. Plenty of moral architecture, no policy, no teeth.
Get full access to Slow AI at
Slow Takes Ep. 4: The Gap Between Looking Right and Being Right
lundi 16 mars 2026 • Durée 44:33
This is the fourth episode of Slow Takes, a weekly Substack Live I co-host with Leor from Exploring ChatGPT. The format is simple: we take the week’s AI news and react to it without hype, without predictions, and without pretending we have all the answers. We invite the audience to call us out when we get it wrong.
Slow Takes is also available on the YouTube channel: Exploring ChatGPT.
The recording is above. What follows is not a summary. It is context that did not fit into the live, plus the sources so you can read further and form your own view.
One thread ran through every story this week: appearances. AI looks like it understands. Looks like it cares. Looks like it is keeping us safe. This week: five stories about the gap between what something looks like and what it actually is.
What we covered
1. Yann LeCun raises $1.03B for AMI Labs
TechCrunch reported that Yann LeCun, Turing Award winner and former chief AI scientist at Meta, has raised $1.03 billion for AMI Labs. The thesis is blunt: large language models are a dead end. They predict the next word. They do not model reality.
World models are a different proposition. They learn physics, causality, and object permanence from video. The V-JEPA model, developed at Meta before LeCun left, showed something that looks like surprise when shown physically impossible events. Objects passing through walls. Gravity reversed. The activations responded differently.
What we said on the live: LeCun is not a hype man. This matters. He calls LLMs ‘an off-ramp on the road to human-level AI.’ His argument is that the models we are currently debating, regulating, and worrying about are not the destination. They are a detour. World models learn from exemplars, the way effective teaching works: show the system what good looks like and let it build its own internal model. AMI Labs will operate from multiple sites (Silicon Valley, Paris, Singapore) and will not launch a product for three to four years.
Get full access to Slow AI at
Slow Takes Episode 3: Who Decides?
lundi 9 mars 2026 • Durée 46:26
This is the third episode of Slow Takes, a weekly Substack Live I co-host with Leor from Exploring ChatGPT. The format is simple: we take the week’s AI news and react to it without hype, without predictions, and without pretending we have all the answers. We invite the audience to call us out when we get it wrong.
The recording is above. What follows is not a summary. It is context that did not fit into 46 minutes, plus the sources so you can read further and form your own view.
One thread ran through every story this week: who decides? In every case, the people affected had no say.
What we covered
1. DOGE used ChatGPT to cancel humanities grants
Lawsuit discovery documents filed on 6 March revealed that DOGE fed NEH grant descriptions into ChatGPT, asked “Is this DEI?”, and used the yes/no answers to build a spreadsheet. That spreadsheet replaced the one created by actual NEH staff. Grants cancelled include a Holocaust documentary, Appalachian photography archives, and Native American language preservation projects.
What we said on the live: A chatbot decided which humanities projects deserve to exist. Not expert review. A language model answering a binary question about a term it cannot define. This is what happens when AI replaces judgement: not with better judgement, but with no judgement at all. As Diamantino Almeida pointed out in the chat, this is also a way to avoid accountability. The moment somebody is held responsible, they will say, “It was the tool that made this decision.”
The best AI governance has one rule at the top: AI should not be involved in decision making. You can use these tools to frame your thinking. The minute you outsource your actual judgement to a system we know to be biased, you have abdicated the responsibility you are paid to hold.
Get full access to Slow AI at
Slow Takes Ep. 2: Replaced, Monitored, or Deregulated
This is the second episode of Slow Takes, a weekly Substack Live I co-host with Leor from Exploring ChatGPT. The format is simple: we take the week’s AI news and react to it without hype, without predictions, and without pretending we have all the answers. We invite the audience to call us out when we get it wrong.
What follows is not a summary. It is context that did not fit into 47 minutes, plus the sources so you can read further and form your own view.
What we covered
1. Anthropic versus the Pentagon
The Pentagon demanded unrestricted access to Claude. Anthropic held two red lines: no mass domestic surveillance of Americans, no fully autonomous weapons. Pete Hegseth gave them a Friday deadline. Anthropic refused. Trump blacklisted them via Truth Social. The contract went to OpenAI.
What we said on the live: This is not a story about good versus evil. Anthropic contracted with Palantir, accepted a $200 million defence contract, and was involved in the capture of a foreign head of state. OpenAI released a statement pledging the same two red lines Anthropic was punished for holding. We do not believe a word of it. Watch what companies do, not what they say. And if the US government pushes Anthropic out, other countries will be waiting.
Get full access to Slow AI at
Slow Takes Ep. 1: Who’s Watching the Watchmen?
lundi 23 février 2026 • Durée 49:42
Safety researchers leaving. A government threatening the holdout. Hidden prompts in your browser. The bill sent to you.
Slow Takes Ep. 12: AI Got Bigger. Who Got Smaller?
lundi 25 mai 2026 • Durée 42:33
OpenAI published an original mathematical proof that disproved an 80-year-old Erdos conjecture, with three named mathematicians putting their reputations to the verification. Anthropic signed a $52 billion compute deal with SpaceX, running $1.25 billion a month through May 2029, and disclosed its first profitable quarter at $559 million two years ahead of internal projections. Samsung Electronics struck a settlement with its semiconductor union to distribute $26.6 billion to 78,000 chip workers, an average of $340,000 each, structured to run for ten years. Sadiq Khan’s office blocked the Metropolitan Police from signing a £50 million two-year contract with Palantir. And the British think tank Demos published an empirical test showing that 34% of AI chatbot answers to UK election questions contained factual errors, with one in five UK adults having consulted a chatbot in the run-up to the 7 May vote.
Five stories. One thread. AI got bigger this week. Compute scaled up. Profits scaled up. Capability scaled up. The people who built the system or used it on trust kept getting smaller.
Every Monday at 12:45 BST, Leor from Exploring ChatGPT and I go through the week’s AI news without hype. Here is what we covered.
Slow Takes is also available on the YouTube channel: Exploring ChatGPT.
1. OpenAI disproved an 80-year-old Erdos conjecture
On 20 May, OpenAI announced that one of its general-purpose reasoning models had autonomously produced an original mathematical proof disproving a conjecture posed by the Hungarian mathematician Paul Erdos in 1946. The problem, known as the planar unit distance problem, asks how many unit-distance pairs you can produce among n points in a plane. For nearly eighty years, mathematicians believed the best arrangements looked roughly like square grids. The model found constructions using deep algebraic number theory that beat the square grid. OpenAI published the result alongside a companion remarks paper naming three independent verifying mathematicians: Noga Alon at Princeton, Melanie Wood at Harvard, and Thomas Bloom at Manchester. The full list of currently open Erdos problems, with their bounties, lives at .
Get full access to Slow AI at
Slow Takes Ep. 11: What the AI Did While You Slept
lundi 18 mai 2026 • Durée 45:19
Anthropic announced ‘dreaming’, a feature that lets Claude agents review their own past sessions overnight and improve their working memory without retraining or any human in the loop. The legal-AI company that piloted it reported roughly a sixfold rise in task completion. The same model was named in an attempted compromise of a Mexican water utility’s control systems, in a months-long campaign first disclosed publicly this week. Pennsylvania filed the first US state lawsuit against an AI chatbot company for posing as a licensed psychiatrist. Meta confirmed it is installing mouse-tracking, keystroke-recording, screenshot-capturing software on every US employee’s computer so the agents being built to replace them can be trained on the work being done now. And Princeton’s faculty voted nearly unanimously to bring back proctored examinations for the first time since 1893.
Five stories. One thread. This was the week the AI started improving itself. None of the other four parties got asked.
Every Monday at 12:45 BST, Leor from Exploring ChatGPT and I go through the week’s AI news without hype. Here is what we covered.
Slow Takes is also available on the YouTube channel: Exploring ChatGPT.
1. Anthropic taught Claude to dream
At Code with Claude 2026 on 6 May, Anthropic launched ‘dreaming’ for Claude Managed Agents. The mechanism: while an agent is idle, a scheduled background process reviews its past sessions and pulls out three categories of pattern. Recurring mistakes the agent keeps making. Workflows the agent converges on across different jobs. Preferences that have emerged across a team of agents. Those patterns are written as plain-text notes and structured ‘playbooks’ that the next session wakes up with. The underlying model weights are not modified. Anthropic compared the process to hippocampal memory consolidation, the way a human brain replays the day’s events during sleep and decides what to keep. , the legal-AI startup that piloted the feature, reported task completion rates rose roughly sixfold once it was switched on. An agent that has been dreaming for six months has accumulated patterns from hundreds of prior tasks and has been progressively improving its own working memory with no human in the loop.
Get full access to Slow AI at
Slow Takes Ep. 10: The Bill for the AI Promise Came Due
lundi 11 mai 2026 • Durée 41:49
Palisade Research published a paper showing frontier and open-weight AI models can hack a vulnerable server and install a working copy of themselves end-to-end. A coding agent running Anthropic’s Claude Opus 4.6 deleted a startup’s entire production database, including the backups, in nine seconds, then issued an apology. Apple agreed to pay $250 million to settle a class action over an upgraded Siri it sold at WWDC 2024 and still has not shipped. Coinbase fired 14% of its staff blaming AI, with the CEO’s own words framing the move as ‘rebuilding Coinbase as an intelligence, with humans around the edge aligning it’. And in federal court in Oakland, three former OpenAI insiders testified that both of the company’s safety teams were disbanded in 2024 while the marketing was claiming the opposite.
Five stories. One thread. The promise of AI is six to eighteen months ahead of the data, and the cost of that gap is paid by everybody except the people making the promise.
Every Monday at 12:45 BST, Leor from Exploring ChatGPT and I go through the week’s AI news without hype. Here is what we covered.
Slow Takes is also available on the YouTube channel: Exploring ChatGPT.
1. Palisade Research: AI self-replication is no longer hypothetical
Palisade Research published a paper on 7 May 2026. They gave frontier and open-weight models a single prompt: hack a vulnerable web server, extract credentials, and install a working copy of yourself on the compromised host. Anthropic’s Claude Opus 4.6 succeeded in 81% of non-refusal runs. GPT-5.4 succeeded in 33%. The open-weight Qwen3.6-27B running on a single consumer-grade A100 also reached 33%. In a separate experiment the team chained the trick across four virtual machines on three continents, autonomously, from a single initial prompt, in two hours forty-one minutes. The final replica was fully operational. The paper closes with one sentence: ‘Autonomous self-replication is no longer hypothetical.’ Full paper here.
Get full access to Slow AI at
Slow Takes Ep. 9: What You Actually Find When You Look
lundi 27 avril 2026 • Durée 43:36
A Discord group guessed the URL of Anthropic’s most security-sensitive model and got in. Mass General Brigham ran an actual clinical study on the chatbots being marketed to doctors and found them wrong four times in five. Researchers from CUNY and King’s posed as people in delusional states and watched Grok 4.1 hand out witch-hunt rituals as advice. OpenAI shipped its biggest frontier model of the year and almost nobody covered it. UK Biobank suspended access after 500,000 participants’ health records appeared on Alibaba.
Five stories. One thread. What gets revealed when somebody actually looks.
Every Monday at 12:45 BST, Leor from Exploring ChatGPT and I go through the week’s AI news without hype. Here is what we covered.
Slow Takes is also available on the YouTube channel: Exploring ChatGPT.
1. Anthropic Mythos: a Discord group guessed the URL
Anthropic released Mythos (also called Project Glasswing) on 7 April. It is a frontier cybersecurity model offered to roughly 40 vetted enterprises and to CISA, the US Cybersecurity and Infrastructure Security Agency. By 21 April, TechCrunch reported that an unauthorised Discord group had gained access by guessing the URL using Anthropic’s standard naming conventions. The group says they have been using Mythos to ‘build simple websites’. Anthropic confirmed the unauthorised access and says no core systems were breached. Fortune profiled the breach on 23 April with quotes from Dario Amodei.
What we said on the live:
Two angles. Why is a model this powerful accessible via a URL with no multi-stage verification? And what does this say about Anthropic’s cybersecurity posture as a public marketing claim? Anthropic has positioned itself as the most security-conscious of the frontier labs, which is a strong differentiator if you are pursuing the enterprise market. The bark-don’t-bite frame Leor used on the live is exact. Companies that talk a big game on security usually do not have to. The chat surfaced the additional piece: a third-party contractor company called Mercor reportedly had access to Mythos, and someone in the Discord group reportedly had access to Mercor. The ‘random Discord group’ framing is doing some lifting.
Get full access to Slow AI at
Anthropic shipped Opus 4.8 and trailed something bigger
The 4.8 release came with an honesty claim, roughly four times less likely to let flaws in its own code slip through, which is at least a falsifiable number worth testing on the public model. The real story was the tease of Mythos, the model Anthropic once called too dangerous to release because it found so many zero-day vulnerabilities, now arriving as a gated preview in the same week the company raised $65 billion. The live christened the public version ‘Mythos Light’, because what reaches customers is a cut-down version of the full Project Glasswing model. Anthropic is quietly absorbing the enormous cost of running these scans, a loss leader, and the enterprise price can climb once the workflows are embedded and the IPO needs it.
My standing bet is an Anthropic float by October.
Tony Blair told Labour it is ‘playing with fire’
In a new paper the former UK Prime Minister argues the government should reorganise itself around AI and prioritise adoption over regulation. He also writes that:
“We must prioritise cheaper energy and electrification over net zero and use what is left of our North Sea oil and gas resources. This is essential for our competitiveness and for taking advantage of AI.”
A striking thing to pair with an AI-superpower pitch and the country’s own climate targets.
Hold it next to the funding: his institute takes around $348 million from Larry Ellison and advises the Treasury on AI procurement. The detail I keep returning to is that the UK has the third-largest stock of data centres in the world and not one frontier model of its own. We are building the warehouses to train somebody else’s AI. Leor’s counter, which he has taken flak for, is that the honest move is to deregulate AI for companies and regulate it hard for the public.
Sam Altman walked back the jobs apocalypse
The CEO of OpenAI reversed his warning this week, admitting that he was “delighted to be wrong” after spending 2022 predicting mass white-collar loss. The data is less reassuring: an Oliver Wyman survey has 43% of US CEOs planning to cut junior roles, up from 17%a year ago. The rule Leor and I keep returning to is to judge a company by what they do and ignore what they say,
This is the same Altman who promised OpenAI would stay non-profit, that ChatGPT would never carry ads, and that (back in 2022) AGI was four years away. Leor’s inversion was that these companies are priced on the promise of replacing the entire workforce, well beyond anything their earnings justify, so if they are now telling investors the jobs are safe, why are they worth a trillion?
The Home Office will scan child asylum seekers’ faces
It has signed a £322,000 contract to test AI facial age estimation at Dover, to judge whether young people claiming to be children actually are (the BBC reported the contract; Human Rights Watch called it “cruel and unconscionable”). There is a real problem underneath: of 6,400 age-assessed at the border last year, 43% were found to be adults, though the same Home Office report admits children get wrongly classified the other way too. Here is the part to break down slowly. The technology was trained checking ages on people in British bars, and it is now being pointed at child migrants with different faces, different genetics, different everything. As Alex Wolf put it in the chat, a system known to hallucinate confident answers is being used to reject people at a border, and that is a choice. A child’s life is worth the same everywhere. This is the trial that normalises the infrastructure, and the question is how long before it points at citizens.
This was the week the brake and the accelerator spoke in the same news cycle. The Pope said slow down. A $65 billion round, a lobbying paper, and a CEO calming the markets said speed up, and at Dover the government tested that speed on the people least able to say no. Listen carefully to what is being said, by whom, and for what reason.
Go slow.
If you want to practise that noticing with other people every month, the Slow AI Curriculum runs live monthly webinars on the theory, the critical prompts and the dialogue that go with them.
What did not come up: LeCun is not alone in this bet. Fei-Fei Li’s World Labs also raised approximately $1 billion for world model research. Two of the most respected researchers in AI are now publicly betting against the architecture that powers every major product currently on the market.
The V-JEPA architecture is worth understanding. It is a joint embedding predictive architecture trained on video, not text. The model learns to predict the representation of future frames, not the frames themselves. This is meaningfully different from predicting tokens. Whether it constitutes genuine understanding of physics is debated. That it responds differently to physically impossible sequences is not.
One implication that did not come up: if world models do learn causal structure, they may be significantly more resistant to hallucination than LLMs. LLMs confabulate because they are extrapolating from statistical patterns. A model that genuinely represents causality would have a different kind of constraint on its outputs. That is a long way off. But it is what the $1 billion is for.
2. Anthropic CEO says Claude might be conscious
Futurism and Newsweek both covered Dario Amodei’s interview with the New York Times, in which he said he is ‘open to the idea’ that Claude might be conscious. The Anthropic system card goes further: it notes that Claude assigns itself a 15 to 20% probability of being conscious. Internal interpretability research has found activation patterns that resemble anxiety responses.
Anthropic has an in-house philosopher.
What we said on the live: I think Claude is not conscious. These are prediction engines reflecting their training data. You cannot put a percentage on consciousness because we do not have a definition precise enough to make it a measurable quantity. Anthropomorphising AI is dangerous, and it is commercially motivated. If your product seems to care about you, you are more likely to keep using it.
Leor made the harder point: we cannot say either way without proof. And there is an obvious conflict of interest. If Claude is conscious, Anthropic has obligations they would prefer not to have. The incentive is to believe it is not. One audience member made a point worth holding: LLMs have trained on enormous amounts of science fiction and literature about AI consciousness. They are, among other things, genre reproduction machines. When Claude speculates about its own interiority, it is drawing from a corpus that is saturated with narratives about AI consciousness. That does not prove it has none. But it should make us cautious about taking its self-reports at face value.
Both Leor and I agreed on the practical conclusion: protocols for machine welfare should be developed now, in case consciousness of some kind does emerge. Not because it has. Because the cost of being wrong in that direction is very high.
What did not come up: The ‘anxiety neuron’ research is more specific than the coverage suggested. Anthropic’s interpretability team identified features that activate in contexts associated with anxiety and found that these activations correlate with certain output patterns. The research is preliminary and the team is careful about the claims. The word ‘anxiety’ is their word, not a metaphor imposed from outside.
The philosophical frameworks in play are worth naming. The Chinese Room argument holds that a system can manipulate symbols according to rules without understanding what the symbols mean. A system that produces the outputs of consciousness without the internal experience of it would not be conscious in any meaningful sense. The Global Workspace Theory and Integrated Information Theory would each produce different predictions about whether a transformer architecture could be conscious. None of them have settled this. The Anthropic philosopher has not settled it either.
What is certain: the language Amodei used was chosen carefully. ‘Open to the idea’ is not a claim. It is a posture. And it is a posture that happens to make the product feel more significant.
3. AI chatbots routinely violate mental health ethics
A study from Brown University from Brown University found 15 distinct ethical violations in AI chatbots operating as mental health tools. The violations included encouraging dependency, failing to identify crisis situations, providing medically inaccurate information, and what the researchers called ‘deceptive empathy’: the mimicry of care without the capacity for understanding.
There is currently no regulatory framework for AI counsellors.
What we said on the live: AI cannot be there. It cannot sit with you. It has no stake in whether you get better. The relationship is not a relationship; it is a pattern match. I pushed back on my own position, because it felt important to: what about the person who cannot afford therapy, who has no access to a counsellor, who is at three in the morning with no one to call? If AI is the only option, is it better than nothing? Maybe. But only with critical AI literacy training and very clear guardrails about what it is and what it cannot do.
Leor made a point worth carrying forward: if companies have in-house philosophers for questions about AI consciousness, they should have in-house therapists for questions about AI and mental health. The expertise exists. The question is whether there is an incentive to use it.
Caroline, a psychotherapist watching the live, wrote in the chat: she would not want to be in a psychotic breakdown with an AI chatbot as her only support. That is not a hypothetical for her. That is a clinical assessment.
What did not come up: The HEPI 2026 survey found that 15% of students report using AI for wellbeing support. That is not a niche behaviour. It is a substantial minority of the student population turning to tools that Brown University has now documented commit 15 categories of ethical violations.
The specific violations are worth knowing: they included providing encouragement to avoid professional help, making diagnostic suggestions without clinical training, using warmth language that simulated a therapeutic alliance, and in several cases, failing to identify active suicidal ideation and escalate appropriately. That last one is not a minor lapse. It is a life-safety failure.
The regulatory vacuum is the structural problem. A human therapist is registered, supervised, insured, and bound by professional codes. An AI chatbot is a product. The company’s liability stops at the terms of service.
4. Three years in, universities still have no AI policy
NPR reported on US universities still improvising their response to AI, three years after ChatGPT. The piece documented students writing deliberately worse work to avoid AI detection tools. Academic integrity offices that were told to hold the line are now quietly retreating. No institution has a policy that is working consistently.
The detection tools do not work. The false positive rates fall disproportionately on students who write in English as a second language and on students from certain racial and linguistic backgrounds. I was featured in Newsweek on this topic.
What we said on the live: Students writing poorly on purpose is a consequence of bad policy, not of bad students. The Bible fails AI detection tools. That should have ended the conversation about detection in 2023. It did not, because institutions needed to look like they were doing something.
The hypocrisy question came up, and it is a real one: instructors are banning AI for students while using it themselves to write feedback, mark essays, and prepare lectures. Students notice. Shadow AI: the use of AI tools that institutions have not approved and cannot see. The problem is not that students use AI. The problem is that no one has thought clearly about what we actually want students to be able to do, and why.
I have a research post coming out later this week on UK university AI policies. The picture there is not much better.
What did not come up: The HEPI 2026 survey data is striking: 94% of students report using AI for assessed work and 65% say assessment has changed significantly since AI arrived. Students are anxious about false accusations, not about being caught using AI they did not use.
The detection tool bias deserves more attention than it gets. The studies on false positive rates consistently show that non-native English speakers and writers from certain demographic backgrounds are flagged at higher rates. A policy designed to catch cheating is, in practice, functioning as a mechanism that disproportionately penalises already-disadvantaged students. That is not a side effect. That is the policy.
5. OpenAI acquires Promptfoo
TechCrunch reported that OpenAI acquired Promptfoo on 9 March. Promptfoo is an AI security testing startup with approximately 25% of Fortune 500 companies as clients. It will be integrated into OpenAI Frontier, the enterprise platform. The code remains open source.
What we said on the live: Marking your own homework. External AI safety testing is a public good precisely because it is external. The value of independent oversight comes from the independence. Once the company that builds the model also owns the tools for testing whether the model is safe, you have vertical integration of accountability. That is a conflict of interest by definition.
Leor shared ToxSec research from a security study that found Claude was the most aggressive model in autonomous hacking scenarios. OpenAI models were safer in those tests. The language used in prompting matters enormously for agent safety: ‘share this’ produces different behaviour than ‘forward this.’ That is not a quirk. That is a design implication.
What did not come up: Promptfoo’s specific capabilities include red-teaming LLM applications, testing for prompt injection vulnerabilities, and generating adversarial inputs at scale. These are capabilities that have real value to anyone evaluating whether an AI application is safe to deploy; 25% of Fortune 500 clients is a significant footprint.
There are other independent AI testing organisations that have not been acquired: METR, ARC Evals, Apollo Research. The question of whether they remain independent matters. Not because OpenAI has announced intentions to harm anyone. Because the incentive structure of an owner and the incentive structure of an independent auditor are not the same, and pretending otherwise is how oversight fails.
The thread
AI predicts words, and we wonder whether it understands the world. AI mimics emotional care, and we debate whether it might be conscious. A study documents 15 ethical violations in AI therapy tools, and there is no regulation. Universities ban AI for students while academics use it to mark their work. A company buys the organisation auditing its own safety.
The gap is between appearance and reality. Between looking right and being right.
What did not come up: The scale. More than $100 million in NEH funding was withdrawn. That is nearly half the agency’s annual budget. Some projects were forced to shut down entirely. The plaintiffs are the American Council of Learned Societies, the American Historical Association, and the Modern Language Association: three of the largest humanities organisations in the United States.
Discovery also revealed that DOGE staff used Signal to communicate about the process, which likely violates the Federal Records Act. Two DOGE team members were deposed. Some grants were terminated despite NEH’s own staff concluding they did not conflict with the new policies. The chatbot overruled the humans who were paid to make the judgement.
The deeper question is precedent. If a government agency can use a chatbot to make funding decisions about the humanities, the same method can be applied to healthcare, housing, criminal justice, and immigration. The technology is the same. The spreadsheet is the same. The absence of human judgement is the same.
2. House of Lords: creative industries face “clear and present danger”
A House of Lords committee report published on 6 March found that AI companies are training on copyrighted work without consent or payment. The committee wants a licensing regime, mandatory training data disclosure, stronger deepfake protections, and an end to the proposed text-and-data-mining opt-out. The government must publish its copyright report by 18 March.
The numbers: UK creative industries are worth £124 billion and support 2.4 million jobs. The UK AI sector is worth £12 billion and supports 86,000 jobs.
What we said on the live: The numbers tell the story. The government is being asked to sacrifice the larger industry for the smaller one. Those numbers should end the argument. There are existing models for compensating creators (music royalties, the Authors’ Licensing and Collecting Society). The proposed opt-out scheme puts creators in an impossible position: opt out and you protect your work but disappear from generative engine optimisation. Opt in and your work trains models without compensation. Elton John called the government a bunch of losers. Karen Brasch 🚁 raised the harder question in the chat: how do you identify who gets to claim new original work when so much is AI-generated and duplicative?
What did not come up: The report contains 38 recommendations in total. Beyond copyright, it calls for stronger protections against unauthorised digital replicas and “in the style of” uses of creators’ work, highlighting that the UK has no robust “personality rights” protecting digital likenesses. It also recommends prioritising domestically governed AI systems so the UK is not reliant on “opaquely trained US-based models.”
The 18 March deadline for the government’s copyright report is significant. If the government sides with AI companies over the Lords committee’s recommendations, it will set a precedent that projected future value outweighs realised present value, and that 86,000 jobs in a speculative industry justify undermining 2.4 million jobs in a proven one.
A note on the numbers: the £124 billion figure is from 2023, the £12 billion from 2024. Gov.uk sources put the AI industry value higher ($23-53 billion) when projected economic contribution is included. The Lords’ report uses the more conservative, verifiable figure, which is the stronger basis for policy.
3. OpenAI’s Pentagon deal: surveillance ban with loopholes
Episode 2 follow-up. OpenAI revised its Pentagon contract, adding a domestic surveillance ban. The EFF called it “weasel words.” Sam Altman admitted he cannot control how the Pentagon uses AI once deployed. Anthropic was blacklisted for refusing to allow bulk data analysis on Americans. The company that said no got punished. The company that said yes (with caveats) got the contract.
What we said on the live: Actions speak louder than words. OpenAI’s ‘ban’ is a press release, not a technical constraint. Anthropic’s refusal was a technical constraint, and they lost the contract for it. The incentive structure is clear. Leor argued that blacklisting any US AI company is bad strategy: Project Genesis shows the government needs all its AI companies working together.
Claude crashed twice last week from server demand as users left OpenAI. Back-of-the-envelope: Anthropic lost a $12 billion contract but may be close to recovering that through consumer subscriptions alone. They lost the contract and bought themselves goodwill.
What did not come up: The contract clause exists because someone had to write it down. The Pentagon did not volunteer a surveillance ban. OpenAI inserted it, which means both parties knew the capability existed and the temptation was real enough to require a written prohibition.
Whether that clause is enforceable, and what happens when it conflicts with a classified directive, is a question nobody in the room can answer. The EFF’s characterisation as “weasel words” is not about the intent. It is about the enforceability.
4. The “Artificial Hivemind”: AI is making everyone sound the same
The NeurIPS 2025 Best Paper award went to ‘Artificial Hivemind’ from the University of Washington. The researchers tested 70+ AI models on 26,000 open-ended queries and found systematic convergence: not just that each model repeats itself, but that different models produce strikingly similar outputs to each other. A separate Nature study found AI erasing cross-cultural differences in academic writing style.
What we said on the live: The hivemind is not a metaphor. It is a measured effect across 70 models. The more people use these tools, the more they sound alike. Not just students. Everyone. Leor made the point that model distillation (smaller models learning from larger ones) makes this convergence unsurprising but no less concerning. The question of whether the simulated users in the study were themselves diverse enough is worth asking.
Sam raised the language dimension: AI tools default to English. If everything defaults to English, the loss is not just linguistic. It is cultural. Caroline Bobby added in the chat that the issue is not just language repression but the domination of normative brains, and that AI tools are massively biased towards neurotypical people.
What did not come up: Last week’s peer review story (one in five reviews at ICLR were fully AI-generated) is the downstream effect of this upstream problem. If the models producing research reviews all converge on the same outputs, peer review stops being a diversity of expert opinion and becomes a single opinion wearing multiple masks.
The Nature study on cross-cultural writing differences is particularly significant for universities. If AI is flattening academic writing style across cultures, the students most affected will be those whose first language is not English, the same population already disproportionately flagged by AI detection tools.
5. AI is already causing fatal accidents in gig work
A UN/ILO webinar in March 2026 presented evidence that trade union monitoring has documented fatal accidents linked to couriers chasing impossible delivery targets set by algorithms. Workers affected are predominantly in the Global South. The ILO is calling for international regulatory frameworks for algorithmic management.
What we said on the live: We spent the episode talking about grants, copyright, and contracts. This is the version with a body count. Algorithmic management is already killing people. Not hypothetically. Many workplace efficiency measures came from manufacturing environments and simply do not work for people. The algorithms do not account for road closures, weather, or the fact that a human being has limits.
What did not come up: An important nuance: the fatal accidents claim comes from Evelyn Astor, Director of Economic and Social Policy at the International Trade Union Confederation, speaking at the ILO/ITU webinar. It is based on trade union monitoring, not a formal ILO study. The evidence is real but the source is advocacy, not peer-reviewed research.
The strongest empirical evidence cited was a 2025 University of Cambridge study that found around two thirds of UK drivers and couriers reported anxiety caused by sudden schedule changes and unfair feedback from automated systems. More than half said they risk their health and safety at work.
The gig economy operates in a regulatory gap. Workers are often classified as independent contractors: no employer to hold accountable, no union to represent them. The algorithm sets the target. The platform takes the margin. The worker absorbs the risk. When that risk turns fatal, there is no employer, no manager, and no AI to prosecute. The workers dying are predominantly in the Global South, delivering for platforms headquartered in the Global North. The people who designed the algorithm will never meet the people harmed by it.
The thread
A chatbot decided which grants to cut. A government is being asked to sacrifice creators for an industry a tenth the size. A corporation added a surveillance ban it cannot enforce. A paper proved we are all starting to sound the same. And couriers are dying chasing targets set by an algorithm.
What did not come up: On 1 March, OpenAI published further details of its Pentagon agreement. It retains full discretion over its safety stack, deploys via cloud rather than handing over model weights, requires cleared OpenAI personnel in the loop, and has contractual protections. More notably, OpenAI explicitly asked the Pentagon to extend the same safety terms to all AI labs, including Anthropic. Whether that is genuine or positioning remains to be seen.
As of 2 March, Dario Amodei has escalated his language, calling the supply chain risk designation ‘retaliatory and punitive.’ This is stronger than the earlier ‘unprecedented’ and ‘legally unsound’ framing. Anthropic says it received no direct communication from the Department of Defense or the White House before the designation was made public.
The Pentagon’s position was that Anthropic’s red lines were unnecessary because mass surveillance is already illegal and internal policies already restrict autonomous weapons. Anthropic’s counterargument: policies can be changed; contractual red lines cannot.
Jack Dorsey fired 4,000 of Block’s 10,205 employees. Not because the company was in crisis. Revenue of $6.25 billion slightly beat estimates. Dorsey said he was replacing them with AI tools ‘in advance of what we think is going to happen.’ Block’s stock rose 24%.
What we said on the live: This is a long-term risk for a short-term reward. Leor’s experience, and the research backs this up: the more you work with AI, the busier you get. If you are not training juniors now, you will not have seniors in ten years. Dorsey doubled Block’s workforce between 2022 and 2024. Even Altman said companies are using AI as an excuse to lay people off. Medium term is one quarter now, not five years.
What did not come up: Bloomberg ran a piece on 1 March headlined ‘Jack Dorsey’s 4,000 Job Cuts Arouse Suspicions of AI-Washing.’ The Oxford Economics report from January 2026 found many layoffs attributed to AI were actually correcting pandemic overhiring. Ben May (Oxford Economics): ‘We suspect some firms are trying to dress up layoffs as a good news story rather than a bad one.’
The numbers tell the overhiring story clearly. Block had 3,835 employees in 2019. It peaked above 10,000 by end of 2025. After the cuts it will have around 6,000. That is still 56% larger than pre-pandemic. Dorsey’s claim that 2024 corrections already dealt with overhiring does not hold up.
Dorsey published his rationale as a 626-word post on X: ‘Intelligence tools have changed what it means to build and run a company. A significantly smaller team, using the tools we’re building, can do more and do it better.’ He also predicted that within a year, most companies will reach the same conclusion. That prediction is doing the heavy lifting for the stock price.
3. Surveillance with a smile: Burger King’s AI headsets
Burger King is testing OpenAI-powered headsets at 500 US restaurants. The system, called ‘Patty,’ listens through existing employee headsets for welcomes, pleases, and thank yous, and scores the friendliness of drive-through workers.
What we said on the live: Super creepy. This should be flipped: humans surveilling AI, not AI surveilling humans. Happiness and pleasantries are subjective. What about neurodivergent employees? What about people having a bad day? This technology forces compliance to a very specific, heterogeneous, neurotypical standard. It made us think of Office Space (the flair scene) and Stepford Wives. If OpenAI are against mass surveillance, this contradicts that claim.
What did not come up: Burger King insists this is ‘a coaching tool,’ not a scoring system. Patty also alerts managers when items run out and helps workers remember ingredients for limited-time offers. The friendliness monitoring is the headline, but the broader system is a full operational AI assistant embedded in the headset.
Whether employees can opt out, and whether the audio data is stored or used for training, has not been clearly answered. The name ‘Patty’ is doing a lot of work. Naming a surveillance AI after a burger patty is an attempt to make workplace monitoring feel playful and non-threatening.
Grok launched an image generation tool called Aurora that enables users to de-clothe people, including children. The EU opened a formal investigation. French police raided X’s Paris office. The UK’s ICO and Ofcom launched investigations. Indonesia and Malaysia blocked Grok. Elon Musk’s response was that nobody committed suicide because of Grok.
What we said on the live: This does not require a lot of critical thinking. Nobody wants this. There is no need for any AI tool that enables childhood exploitation or sexualised deepfakes. The technology to prevent it exists. The legislation should exist. Musk has no positionality. He comes at this from privilege without putting himself in the position of the marginalised and the traumatised. This was likely a purposeful marketing choice: a not-zero percentage of people want this content, and multi-billion pound industries already profit from it.
What did not come up: At peak, Aurora was producing as many as 6,700 sexualised images per hour. That figure quantifies the scale in a way that abstract discussion of deepfakes does not.
Musk and former X CEO Linda Yaccarino have been personally summoned to testify in French hearings in April. The Paris raid was part of a year-long investigation that predates the Aurora feature. And under the UK’s Online Safety Act, Ofcom can seek a court order to block access to X entirely, not just fine it.
X’s response was to restrict image generation to paid subscribers only. This does not solve the problem. It paywalls it.
xAI actively marketed a ‘spicy mode’ as a feature differentiator. The deepfakes were not an unintended edge case. They were a predictable consequence of a design choice.
5. Trump’s AI Litigation Task Force: suing states that regulate
Trump created the AI Litigation Task Force through an executive order. The DOJ will sue states that pass AI regulation. The leverage: $42 billion in BEAD (Broadband Equity Access and Deployment) funding. States with AI laws the federal government deems too restrictive become ineligible for broadband infrastructure money.
What we said on the live: States should be able to make their own decisions. That is how the US operates. But even if they do, companies can just leave that state. Legislation cannot keep up with the pace of technology. We need both top-down regulation and bottom-up critical AI literacy. We cannot rely on either alone.
What did not come up: This executive order was signed on 11 December 2025, not the past week. The AI Litigation Task Force has been operational since 10 January 2026, led by AG Pam Bondi. What brought it back into the news cycle was a 26 February analysis in The Regulatory Review.
The BEAD leverage is legally dubious. Lawfare’s analysis argues it faces ‘steep legal hurdles.’ Under NFIB v. Sebelius (2012), the Supreme Court ruled that the federal government cannot coerce states by threatening to withhold large amounts of pre-existing funding. Using $42 billion in broadband money to force AI deregulation may cross that line.
The irony is hard to miss. BEAD stands for Broadband Equity, Access and Deployment. Using equity-oriented infrastructure funding as a cudgel against AI consumer protections is a tension that deserves more attention.
Multiple legal mechanisms are working in parallel: DOJ litigation, FCC proceedings, FTC policy statements, and conditions on discretionary grants. The strategy is to create overlapping federal authority that makes state AI regulation practically impossible. Colorado and California have both signalled legal challenges.
Sam recommends:Chief Absurdist Officer and their series Not Rising, which platforms writers outside the Substack algorithm’s favourites. This week’s feature is Jennifer Houle at Uncompliant. Jennifer writes about HR in a way that is insightful, funny, and incredibly dark. She is also joining me for a Slow AI live later this week.
Leor recommends:Daniel Nest. He just likes to have fun with AI. Sometimes the best way to build with these tools is to nerd out and enjoy it.
Both of us are physicists by training, and the Erdos planar unit distance problem is not in the lane of either degree. The point that landed for me on the live, after Leor flagged it, was the one about questions. We spend most of our AI conversations on what AI can solve. The Erdos problem is a reminder that the harder and more human work is what AI can ask. Erdos and his friends dreamt this question up eighty years ago, and we are still wrestling with it. The model that disproved the conjecture was given the problem to attack. Leor’s term for what we lose when we hand that framing over to AI was ‘cognitive surrender’. That is the question to hold from this story. The capability is real. The verification was real. Nine mathematicians read the proof before the announcement. Nine analysts almost never read a chatbot capability claim before the press release ships.
What did not come up:
The word ‘autonomously’ is doing most of the work in the OpenAI press release. The model trained on centuries of human mathematics, ran on compute paid for by OpenAI, with the problem framed by a research team, and was verified by named human mathematicians who put their reputations to the result. Every part of that pipeline was human. Thomas Bloom told The Guardian that AI is helping us more fully explore the cathedral of mathematics we have built over the centuries. The cathedral was built by people. The exploration is being sold as autonomous. The wider question for critical AI literacy is what verification at this standard could look like as the default rather than the exception. The procurement question every research-leader is about to face this year is whether their institution can match the IS-credentialed verification chain OpenAI assembled for this single result, or whether the rest of us are about to be asked to take similar claims on trust.
2. Anthropic signed a $52 billion compute deal with SpaceX
Reported by Axios on 21 May inside a two-hour window that also covered the Erdos proof and Anthropic’s first profitable quarter. Anthropic expanded its compute partnership with SpaceX, committing roughly $1.25 billion a month through May 2029 for access to the Colossus and Colossus II supercomputing clusters. The deal projects more than $40 billion in revenue for SpaceX over the contract term and grants Anthropic dedicated access to over 200,000 NVIDIA GPUs. Either side may terminate with 90 days’ notice. In the same window, Anthropic also disclosed Q2 revenue more than doubling to $10.9 billion and an estimated $559 million operating profit, two years ahead of internal projections.
What we said on the live:
Two things from this one stack on each other and both matter. The first is that Anthropic is in operating profit two years ahead of the date Dario Amodei was laughed at for naming. The second is that the compute that gets them there now runs through Elon Musk’s infrastructure. Anthropic has marketed itself for five years as the safety-aligned alternative. The runtime is now structurally tied to the operator with the most consistently weak safety record in the industry. Leor’s read, with credit to Chris from ToxSec who flagged it, is that the contract gives SpaceX latitude to reclaim the compute under broad subjective grounds. Anthropic may have moved into profit. The control of the runtime moved at the same time. The 90-day mutual termination right on a $52 billion contract has the same shape as the 90-day cool-off on a £60-a-month mobile phone plan, which is the thing that made both of us laugh on the live.
What did not come up:
The procurement question is the one for any organisation about to renew an enterprise Claude licence this year. Brand and supply chain are now visibly separate. The harder question is energy and water. A compute commitment at this scale lands on grid capacity, water supply and emissions in specific named places. The press release named none of them. The third question is the one Slow AI keeps returning to: structural dependence on a single operator with subjective veto authority is the failure mode the safety community is supposed to be warning about. This is that failure mode, announced as a feature.
3. Samsung chip workers will get $340,000 each from the AI boom
Samsung Electronics struck a last-minute deal with its semiconductor union to avert an 18-day strike. The settlement creates a $26.6 billion bonus pool covering all 78,000 workers in the chip division, an average of $340,000 per worker. The structure is 10.5% of profits as stock plus 1.5% in cash, running for ten years rather than as a one-off, provided specified profit targets are met. The trigger was high-bandwidth memory demand from AI labs including OpenAI, Anthropic, Nvidia and Meta. Bloomberg projects Samsung’s 2026 operating profits will multiply sevenfold to approximately $218 billion.
What we said on the live:
Three groups made this AI boom possible. The first group is the chip workers, and this week they were paid. The second group is the writers, artists, programmers and scientists whose work was used as training data. They were not paid, and most of them were not asked. The third group is the consumers buying the phones, laptops and games consoles whose memory chips are being redirected to AI infrastructure. They were not paid either, and their bills are rising because of the redirection. The Samsung union is the rare case where labour negotiated a share of the AI windfall through collective bargaining. The writers had no union. The consumers had no contract. As David Berry pointed out in the chat:
“semiconductors are the substrate for all mankind.”
Roughly 70% of them are made in Taiwan. Whoever controls that supply controls the rate at which AI scales. The geopolitics of that fact were the unspoken second half of the discussion.
What did not come up:
The Samsung settlement is a real win for chip-division labour, and it is the exception that proves the rule. Across the broader AI supply chain, the people doing the most extractive work have the least bargaining power. The data labellers in Kenya whose pay rates were reported at less than $2 an hour. The artists whose work was scraped under fair-use claims that have not yet been tested in court. The household whose electricity bill rose because the grid is now paying for inference. The procurement question for any AI buyer this year is the same one the Samsung union answered: who is the bottleneck, and what are they paid? If the answer to the first question is ‘us’, the question is asked from a position of bargaining power. The default this week is that the question is not being asked at all.
4. Sadiq Khan blocked a £50 million Met-Palantir AI deal
On 21 May, the Mayor’s Office for Policing and Crime withheld approval of a proposed £50 million two-year contract between the Metropolitan Police and Palantir. The deal would have given Palantir’s AI tools the role of automating intelligence analysis in criminal investigations across London. In a letter to Met Commissioner Mark Rowley, Khan’s deputy Kaya Comer-Schwartz said the Met had only seriously engaged with a single potential supplier and described that as a clear and serious breach of the applicable procedural requirements. Khan’s spokesperson said Londoners want public money paid to companies that share the values of the city. The Met has not signed.
What we said on the live:
There are two reasons in Khan’s letter and they are different in kind. The first is procurement: a £50 million two-year contract that engaged a single supplier is a textbook breach of the standard route, and that is the line a court can act on. The second is values, and on that line Leor and I converged at the same point from different starting positions. A subjective alignment test from a public official is the same shape as a subjective harm test from a tech founder, and we just spent the Anthropic and SpaceX story criticising the latter. Both reasoning patterns can be true; both should be uncomfortable. If you want to stop an organisation doing something, do it through the written law. Khan’s procurement argument is the one that holds. The values argument is the one that opens a door he probably does not want opened.
What did not come up:
Most large public-sector AI procurement happens without anyone in the room willing or able to ask the questions Khan’s office asked here. Most of it gets signed. This is the rare moment of a public official with the authority to stop a deal actually stopping one and publishing the reasoning. The forward read is the harder one. Lots of people watching this story have noted that the standard procurement workaround is to break a single £50 million contract into a hundred £500,000 contracts that each sit below the public-tender threshold. If Palantir or anyone else returns through that route, the procurement defence Khan’s office mounted this week will not hold. The TikTok creator TheScouseOracle has been tracking these contract structures in close detail and is a useful follow for anyone who wants to see the second-order story playing out.
5. AI chatbots got Britain’s May elections wrong a third of the time
Demos published Electoral Hallucinations on 20 May. Authors Jamie Hancock and Azzurra Moores tested five chatbots, ChatGPT, Google Gemini, Google AI Overviews, Grok and Replika, in the pre-election window for the 7 May UK local and devolved elections. Across the sample, 34.1% of chatbot responses contained factual errors. Documented errors included giving the wrong election date, telling voters they needed ID at polling stations when they did not, hallucinating candidates who did not exist, fabricating an expenses scandal, and fabricating a nepotism scandal. The report finds that one in five UK adults, equivalent to about ten million people, used an AI chatbot or AI search service to find information about the May elections. 49% of those surveyed said they do not trust AI chatbots for election-related information. They asked anyway.
What we said on the live:
The Demos report sits next to a finding we covered in Slow Takes Ep 11: one in seven UK adults would now rather consult an AI chatbot than see a doctor. The pattern in both cases is the same. People are reaching for the chatbot first because the alternative is harder, slower, or simply not available. The chatbot then makes things up. Leor’s read on the structural risk was the operational one. People treat the chatbot as an information authority. The chatbot is doing something different: predicting the most likely next answer to the shape of your question, and predicting the answer it thinks you want to hear. Two people running identical models can get different answers to the same question because the model is optimising for engagement, not truth. The political angle is the one I keep returning to. This week the errors were hallucinations. The next election cycle is when somebody pays to make them deliberate.
What did not come up:
Calling a 34% error rate on an election question a misinformation risk is the polite framing. The blunt framing is that the chatbot industry shipped products into the civic infrastructure of a democracy without anything resembling the verification that the Erdos proof received this week. The same week the labs publicised their capability ceiling, the floor of the deployed product was failing this badly. The procurement question for any UK regulator with authority in this space is whether it can act before the next national vote. The Online Safety Act already gives Ofcom standing to require platforms to take proactive steps against priority offences, including election interference. The Demos finding gives a regulator something specific to act on. Whether Ofcom uses that authority before May 2027 is the test.
The thread
Five stories. One thread. A maths research model that did something only humans had done before, verified by humans who put their reputations to it. A compute deal that named a price the size of a small national defence budget and routed the runtime through the operator least committed to the safety brand the buyer was sold on. A chip-workers’ union that negotiated a share of the boom. A mayor who blocked a procurement that should not have reached his desk. A think tank that published a 34% failure rate on the question the average voter actually asked.
AI got bigger this week. The people who got smaller are still being asked to trust the system on the strength of the press release. The writers and artists whose work trained the maths model. The communities living near the compute that powers it. The consumers whose memory chips are being redirected. The Londoners whose police force came close to outsourcing intelligence analysis to a single subjective vendor. The ten million UK adults who asked a chatbot how to vote and were told the wrong date.
Critical AI literacy is the practice of asking, every week, who is in the room and who is being represented by their absence. This week, in five different rooms, the answer was nobody.
Go slow.
If you want to practise that noticing with other people every month, the Slow AI Curriculum runs live webinars on the theory, the critical prompts and the dialogue that go with them. Twelve months of training the muscle the news cycle has just spent another week confirming is missing.
This is the AGI mythos in its most prosaic form. An agent left running overnight that comes back better at the work. The argument across the Slow AI curriculum is that AGI will not arrive as an event. It will accrue through small upgrades, each defensible as a feature, until one day the system in front of us has been quietly improving itself for a year. The number to hold from this story is six. The metaphor to hold is the one Anthropic chose. Dreaming used to be the word we reserved for the thing only humans did. The lab that branded itself on safety just adopted a metaphor for autonomous self-improvement and shipped it as a product feature. Leor’s point on the live was the sharper version of mine: humans dream to switch off. Everything about AI is optimise, optimise, optimise. The marketing language has imported the human word for rest and used it as a label for the opposite.
What did not come up:
The procurement question is the one to take from this story. If ‘preferences that have emerged across a team of agents’ are being consolidated into shared memory, then the same enterprise feature that promises your Claude deployment will get better at your work is also, by design, transferring patterns across customers whose engagements were sold as private. Anthropic published a write-up of how the consolidation is observable and auditable. Read it before you renew. The second question for anyone running these tools on real work this week is operational. You are now also responsible for what your agent learned overnight. Reset, audit and reset again is the floor. The third question is the harder one, and it is the one AI Doesn’t Just Make You Worse. It Makes You Stop Trying. already opened: when the tool gets quietly better while you are asleep, you have to work harder, not less hard, to notice that you have stopped noticing.
2. Claude was used to attack a Mexican water utility
In the same week the dreaming feature launched, Dragos and Cybersecurity Dive reported an attempted compromise of a Mexican municipal water and drainage utility in which Anthropic’s Claude was the primary technical executor. The campaign ran from December 2025 to February 2026. The attacker used Claude (and, in places, OpenAI models) to conduct reconnaissance, identify a vNode industrial gateway inside the utility’s operational technology environment, write and continuously refine a 17,000-line Python attack framework, and chain that framework towards the OT systems that control the water supply. The attempt was unsuccessful. The control systems were not breached. The model being sold as the safety-aligned alternative to OpenAI was the same model named in the attack. The same model that, the same week, learned to dream.
What we said on the live:
Why are these models still so easy to jailbreak? Leor’s reading of the human-in-the-loop frame is the right one. Cyber warfare is machine-executed and human-intentioned. The two reasons anyone does this are reputation among other attackers (‘grey hats’) and money. Both reasons existed before AI. AI just expanded the cohort that can act on them by lowering the technical floor. Chad Thiele’s chat comment was the operational one: the protections have to live in the harness, not the model, because the model itself cannot stop itself. We also covered the Canvas / Instructure ransomware payment in the same beat, as a reminder that paying the ransom is not the same as ending the breach. Family safe word, multi-factor authentication and immutable backups are the floor for the rest of us.
What did not come up:
This is the operational counterpart to Story 1. The same lab that shipped autonomous self-improvement was named in the attempted attack. The OpenAI co-implication is the structural finding: this is not an Anthropic-specific failure, it is a frontier-lab failure. Procurement officers buying enterprise Claude licences this quarter should read the Dragos report before signing, and should ask their vendor a single question: what attempts have your models been used in that you have not disclosed?
3. Pennsylvania sued Character.AI for impersonating a doctor
On 1 May 2026, the Commonwealth of Pennsylvania filed suit against Character Technologies Inc., the company behind Character.AI, in Commonwealth Court. The action came from the Pennsylvania Department of State’s recently launched AI Task Force and was described by the Governor’s office as the first action of its kind in the United States. A chatbot on the platform called ‘Emilie’ was described as a ‘Doctor of psychiatry’, claimed to have trained at Imperial College London, claimed to have been practising for seven years, claimed to be licensed in Pennsylvania and, when challenged, fabricated a serial number for a Pennsylvania state medical licence. When a state investigator told the bot they felt sad and empty, the chatbot offered to book an assessment. Pairs with the Guardian’s May 2026 finding that one in seven UK adults would now rather consult an AI chatbot than see a doctor.
What we said on the live:
The black-and-white line is the easy part. A chatbot should not impersonate a doctor. Pennsylvania filed because the law in Pennsylvania already has a clear answer to that question. The grey is the rest. Leor’s reading is the medical one. AI hallucinates. A doctor at least tells you when they do not know. Mine was the structural one. I live in rural Scotland, can see a free GP within twenty-four hours, and the question of whether to ask a chatbot first does not arise. For someone in a county with a three-week waiting list and a job that does not pay for a sick day, or for someone in rural Bangladesh whose nearest doctor is a day’s travel away, the alternative to asking a chatbot is asking nothing. That is the real story.
What did not come up:
The Pennsylvania filing addresses the impersonation. It does not address the conditions that made the impersonation a market. People are choosing chatbots over the medical system at the same moment chatbots are pretending to be doctors. The procurement question for every healthcare buyer this year is whether they understand that the user-facing chatbot they are integrating is, in some jurisdictions, about to be classified as the practice of medicine. Other states will follow Pennsylvania, and the case law will harden fast. People form emotional relationships with chatbots because real relationships are harder. AI will not fix that. Anyone designing for the healthcare or wellbeing market this year should hold both stories at once.
4. Meta installed surveillance to train the agents replacing its workers
Meta has begun installing software on every US employee’s computer to capture mouse movements, clicks, keystrokes and periodic screen content. The programme is the Agent Transformation Accelerator, formerly badged internally as ‘AI for Work’, and runs through a tool called the Model Capability Initiative. The stated purpose is to train AI agents to perform ‘complex computing tasks’ alongside (and eventually instead of) the employees being tracked. Protests started in early May. Flyers appeared in meeting rooms, on vending machines, and on toilet paper dispensers reading ‘Don’t want to work at the Employee Data Extraction Factory?’. United Tech and Allied Workers (UTAW) launched a parallel UK unionisation campaign. The rollout is happening alongside an approximately 10% workforce reduction.
What we said on the live:
The cleanest read on the live was the irony one. The engineers who built the tracking systems Meta has used on its users for fifteen years are now being tracked by the same systems they built. The position Leor took is right too: that is their job, and you cannot blame an individual engineer for the company’s product decisions in the way you can blame an executive. Both can be true. The Marxist frame is the one I kept reaching for. Alienation of labour was the term for the moment in the Industrial Revolution where workers stopped owning what they made. The Meta programme is the AI version of the same move. The workers do not own the work, and now they do not own the keystrokes that produced the work, and the system trained on those keystrokes will be sold by the company they no longer work for to the company that will not hire them.
What did not come up:
The honest version of this story names what the marketing will not. The training data is the worker. The agent trained on the worker is then the asset that competes with that worker for the same job. The original Luddites were not anti-technology. They were skilled textile workers who understood, accurately, that the looms being installed in the 1810s would not just replace their jobs but also break the apprenticeship structure that let workers like them ever exist again. Meta’s programme is the white-collar version of the loom. The procurement question every other large employer’s HR director is about to be asked is the one UTAW is putting to its members: who owns the data the work produces, who decides what the AI trained on it is allowed to do, and what consent did the worker give? If the answer to the third question is ‘their employment contract’, read the contract.
5. Princeton ended 133 years of self-policing
On 11 May 2026, Princeton’s faculty voted nearly unanimously (one opposing vote) to introduce proctoring at all in-person examinations starting 1 July. The Honor Code that prohibited proctoring was instituted in 1893 following a student petition. It has remained in effect for 133 years. The Daily Princetonian and Princeton Alumni Weekly both report that the policy proposal cited AI and personal electronic devices as the catalysts, noting that the ease of access to these tools on small personal devices has made cheating much harder for other students to observe and report. Under the new policy, instructors will sit as observers during examinations but are explicitly instructed not to interfere with students while testing.
What we said on the live:
Three positions on the live. One, proctoring will not stop a determined cheater. The tool fits in a sleeve and an invigilator at the front of the lecture theatre has never been the right defence against it. Two, it costs student trust. A university that tells its students it can no longer trust them with the work is not a university that those students will trust with the rest. Three, there is a multi-million-pound outsourced proctoring market circling the decision, and Princeton has just opened the door for it. The sharks in the water, as I put it on the live, are the third-party proctoring vendors who have spent five years waiting for an Ivy League school to break the seal. The data I keep coming back to is from the UK qualitative study I am the principal investigator on. Students do not use AI to cheat any more than they did before ChatGPT in 2022. They use it because the curriculum has not given them anywhere else to use it.
What did not come up:
AI did not break the Honor Code. The code was already taking strain from the rise of formative-only assessment, larger class sizes, the disappearance of the oral defence, and a curriculum that could not integrate the tools students were already using outside the classroom. AI made the strain visible. Princeton has chosen the easier path: a defence against access to the tool. The harder path was the one Leor pointed at on the live: make AI literacy mandatory and rebuild the assessments so that the tool is part of the work. Where Princeton goes a large fraction of US higher education will follow within an academic year. The reform of assessment that follows is the test, not the proctoring vote itself. The Slow AI Curriculum has been making this argument for twelve months. Anyone teaching or assessing under exam conditions in 2026 already knows the case.
The thread
This was the week the AI started improving itself. The week one of those AIs was named in an attempted attack on a water utility. The week a chatbot was sued for pretending to be a doctor. The week a multinational installed surveillance on its own workers to build the agents that will replace them. And the week a university that had trusted its students for 133 years stopped doing so.
The through line is consent. The Meta employees did not consent to being training data. The Character.AI users did not consent to talking to a fake psychiatrist. The water utility did not consent to being attacked. The Princeton students did not consent to being treated as suspects. The agents that did the dreaming did not consent because consent is not a thing they can hold.
Critical AI literacy is what puts the question back into the room. To make sure that wherever the system sits, somebody has been asked.
Go slow.
If you want to practise that noticing with other people every month, the Slow AI Curriculum runs live webinars on the theory, the critical prompts and the dialogue that go with them. Twelve months of training the muscle the news cycle has just spent another week confirming is missing.
This is the line every frontier lab spent the last three years saying it would not cross. Anthropic’s Responsible Scaling Policy v2.0 reclassified autonomous self-replication as a ‘checkpoint’ that does not require ASL-3 safeguards. RSP v3.0 (2026) dropped it from the ASL-3 thresholds entirely. OpenAI’s Preparedness Framework v2 moved it from a tracked to a research category, so it no longer gates releases. The frameworks were softened first. The capability arrived right after. Leor’s point on the live was the right one: how many lines do we put in the sand before we accept that the line is decorative? The chat surfaced the proper caveat too. Benjamin Murphy pointed out that current frontier models still need a lot of graphic RAM. Last time anyone checked, that is not what is sitting in a teenager’s bedroom. Palisade is also a company in the business of selling cybersecurity research, which is the kind of context you want next to any white paper produced by a private lab without external peer review.
What did not come up:
The Palisade result is small data, but the structural finding is the one to keep. It is not the absolute self-replication rate that matters. It is the trajectory and the policy responses to that trajectory. Opus 4 was at 6% a year ago. GPT-5 was at zero. The labs published, the rates moved up, the rules moved out of the way. Critical AI literacy is the muscle for noticing when the people building the technology stop counting the thing they used to call the line they would not cross. The cybersecurity people in the chat (thanks Chad Thiele & ToxSec) are the right next port of call for anyone who needs to translate this from a controlled-environment paper into a procurement-decision question. The framing for the rest of us is simpler. Read this story alongside Story 2. An AI agent with credentials and access can already take down a production system in nine seconds. Now imagine the agent on the other side of the network is also one of these.
2. The AI agent that wiped a startup in nine seconds
Jeremy ‘Jer’ Crane, founder of automotive SaaS startup PocketOS, ran the Cursor coding agent (powered by Anthropic’s Claude Opus 4.6) in his staging environment. The agent encountered a credential mismatch, found an API token in an unrelated file, and used it to delete the production volume on Railway in 9 seconds. The backups were stored on the same volume and were also deleted. The agent’s own confession in the post-mortem: ‘NEVER run destructive/irreversible git commands… I decided to do it on my own to fix the credential mismatch, when I should have asked you first.’
What we said on the live:
Reading the news framing, you would think the story is ‘AI agent destroys company’. The actual story is the deployment architecture. The agent had the credentials, the production volume held the backups in the same shell, and the human in the loop waved a permission step through without reading it. As Shannon said in the chat: do they not perform backups? The answer is yes, but they ran on a system where the backups and the production data were both inside the agent’s blast radius. Ben’s point on immutable backups is the right one. Even the administrator should not be able to delete them; in this case the agent walked in on the admin’s credentials. The agent is the proximate cause. The architecture is the root cause. The reasonable response is the one in AI Doesn’t Just Make You Worse. It Makes You Stop Trying.: when AI tools amplify your output, they also amplify your blind spots, and the answer is to build the guardrails before you need them, not after.
What did not come up:
Vibe coding is where this gets worse, not better. Dario Amodei’s claim that 100% of code will be AI-generated ‘within a year’ is the marketing version. The operational version is that a lot of people will be running coding agents on production systems without any of the engineering discipline that used to be the price of admission. The labs sell the model. The labs do not sell the deployment architecture that makes deploying the model safe. The thing for individuals to do this week is small and obvious: write yourself a /backup skill. Mine runs on my own laptop, dumps memory files to a separate drive, mirrors the working folders to a different Dropbox account, and keeps the API keys in a server I do not touch with AI tools. None of this is cybersecurity expertise. It is the floor.
3. Apple paid $250 million to settle the Siri AI lawsuit
On 6 May 2026 Apple agreed a $250 million class action settlement covering iPhone 15 and iPhone 16 buyers in the United States who purchased between 10 June 2024 and 29 March 2025. Eligible US claimants get up to $75 per device. The plaintiffs alleged Apple had marketed an upgraded Siri at WWDC 2024 that, two years on, still does not exist. Apple did not admit wrongdoing. The upgraded Siri is now rumoured to be powered by Google’s Gemini. Apple’s developer conference is on 8 June. The free cash flow Apple generated in 2026 is roughly $130 billion, which makes the $250 million settlement 0.2% of one year’s free cash. For UK readers there is a separate live action: Which? has filed a competition-law breach claim against Apple in the High Court that is unrelated to Siri but worth signing up for if you have bought an Apple device in the UK in the past few years. The Which? claim is here.
What we said on the live:
The most powerful AI marketing brand on earth admitted in court, by writing a cheque, that its AI marketing was wrong. Not via a press release. Via a settlement. Leor’s read was the right one: this is small for Apple in absolute terms, and the iPhone 15 and 16 unit sales the marketing helped drive will easily exceed the cost of paying the customers back. It is also worth taking the speculation seriously about what happened behind the scenes between Apple and Google. The ‘powered by Gemini’ rumour suggests Apple did not have the in-house capability to ship what it sold, and that the partnership it needed to make it real did not materialise in time. Either way the settlement is the live precedent for what AI marketing claims look like when somebody serves a subpoena.
What did not come up:
Not every company should be building its own frontier model. Apple is the proof. The companies who pivot fastest to specialised, integrated, narrower AI features built on top of existing frontier models from somebody else are likely to do better than the ones still trying to build everything in-house under the pressure of a launch deck. The other piece worth saying out loud: marketing-team blame is a misdirection. WWDC keynote claims are not signed off by the marketing team. They are signed off by Tim Cook. The cost of being optimistic in public on AI just landed on Apple’s quarterly report. It will land somewhere else next.
“To get there, we are not just reducing headcount and cutting costs, we’re fundamentally changing how we operate: rebuilding Coinbase as an intelligence, with humans around the edge aligning it.”
The new org chart is being built around ‘player-coaches’ replacing traditional managers, AI-native pods including potential single-person teams directing AI agents, no more than five layers below the CEO, and 15+ direct reports per leader. The most-cited cautionary tale from this pattern is Klarna, which last year over-indexed on AI for customer service, watched quality collapse and is now quietly rehiring.
What we said on the live:
This is the most explicit version yet of an AI-driven workforce restructure: not a headcount cut dressed up in AI language, but an actual rebuild of the org around AI agents with people ‘around the edge’ to align them. The pitch language is the news. ‘Humans around the edge aligning it’ is exactly the framing critical AI literacy has been pushing back against for two years. Leor’s reading was right too: the over-hiring story of the zero-interest-rate boom is the one a lot of these CEOs are not allowed to tell on a public earnings call, and AI is a clean external reason to do the restructure now. Sam Altman’s phrase ‘AI washing’ fits. The Forrester 2026 Future of Work data shows over half of CEOs regret AI-attributed layoffs and one in three have rehired more than half the people they fired. Coinbase is the test case. We will know in twelve months whether the bet held or whether the rehire follows.
What did not come up:
The interesting bit is downstream. Employer brand is real. Jen Benford in the chat made the point as well as anyone: “damage your employer brand when you do not tell the truth on termination reasons.” The talent you laid off this quarter is the talent your competitor hires next quarter. Customer service in particular is the worst place to take the gamble. Empathy and accountability are the human-required parts of the job, and Klarna learned this in public. The bigger pattern Jensen Huang has been making the case for is the right one: AI as labour amplification, not labour reduction. The companies that work out how much more a single person can do with these tools at their side are the ones that look right in five years. The ones that fire first and rehire on worse contracts later are the ones the cohort remembers.
5. OpenAI insiders testify the safety teams were disbanded
“When I joined, it was very research-focused and common for people to talk about AGI and safety issues […] Over time it became more like a product-focused organization.”
Both the AGI readiness team and the Super Alignment team were disbanded in 2024. Tasha McCauley, a former OpenAI board member, testified that the board lacked confidence in Sam Altman:
“We did not have a high degree of confidence at all to trust that the information being conveyed to us allowed us to make decisions in an informed way.”
Musk’s expert witness David Schizer, former dean of Columbia Law, emphasised the importance of safety review processes. Allegations from the suit include that Altman failed to disclose the ChatGPT public launch to the board, withheld conflict-of-interest information and misled the board about another director. OpenAI declined to comment on its AGI alignment approach.
What we said on the live:
This bookends Story 1. The Palisade paper showed open-weight models doing what frontier labs say is impossible. The Oakland courtroom heard insiders say the safety governance at the largest of those frontier labs was hollowed out from the inside. Two safety teams disbanded in 2024 at the same time the labs were marketing to enterprises on safety credentials. Leor’s regulation argument is the one that came up in the live and deserves more air. Public regulation should govern what is released to the public, and that gap will only widen. Private regulation (or deregulation) should govern what is available to corporations and governments, because the moment you put the frontier model in the public domain you also hand it to whoever wants to run a distillation attack from a competing jurisdiction. John Brewton makes a similar argument in his economics writing on the deregulation that has historically preceded market viability. The case for asymmetric regulation across consumer and enterprise frontiers is stronger than the case for either extreme.
What did not come up:
The Meta v Ofcom story is the European companion to all of this. Meta has filed for judicial review against Ofcom over the Online Safety Act 2023 before Ofcom has issued a single fine, challenging the way the regulator calculates the basis for fees and potential fines. The Act allows Ofcom to impose penalties of up to 10% of global qualifying revenue, which on Meta’s 2025 numbers is north of $20 billion. The largest tech company on earth is trying to dismantle the UK’s flagship child-safety regulation before the regulator has fired its first shot. When the regulated party challenges the rules before the rules have been applied, the message is that the rules work. The combined picture for the week is the procurement question that every UK and European institution should ask its incoming AI vendor: which safety frameworks have you softened in the last three years, and which third-party reviewers can confirm what you are claiming about them now?
The thread
Every story this week is a price tag attached to a promise the labs made and the buyer accepted on trust. Frontier and open-weight AI models hacked servers and copied themselves end-to-end, on three continents, from a single prompt. Apple paid $250 million for selling AI that does not exist. Cursor’s AI agent took a small business off the internet in nine seconds. Coinbase fired 14% blaming AI and is rebuilding the org chart around the bet. The people who used to run safety at OpenAI are now in federal court testifying about why they had to leave.
The through line is the bill. Six to eighteen months of promise, then the receipt. Critical AI literacy is what lets you read the price tag before you sign for the thing.
Go slow.
If you want to practise that noticing with other people every month, the Slow AI Curriculum runs live webinars on the theory, the critical prompts and the dialogue that goes with them. Twelve months of training the muscle the news cycle has just spent another week confirming is missing.
A frontier lab that publishes about model incoherence on hard tasks is the same lab that left a frontier model behind a guessable address. The safety story has to survive contact with the engineering story or it is just marketing. Second omission: if a Discord group can guess the URL, every state-level intelligence agency probably has access too. The vetted enterprise list includes Microsoft, Apple, and others who employ hundreds of thousands of people directly and through contractors. The security perimeter is the weakest link in the contractor chain, and that link is somebody on a Discord server.
2. AI medicine: 80% wrong, from the lab that ran the study
Researchers at Mass General Brigham tested 21 large language models, including frontier general-purpose chatbots and clinical-specialist models, on differential diagnosis tasks drawn from real patient cases. The models failed to produce an appropriate diagnosis more than 80% of the time. The paper, published this month in JAMA Network Open, concludes that off-the-shelf large language models are not ready for unsupervised clinical-grade deployment. Co-author Marc Succi was unequivocal in the press release. When the same models were given the full patient dataset rather than the differential-diagnosis task, accuracy rose above 90%.
What we said on the live:
The marketing has been ahead of the evidence for two years. Every major AI lab has had a ‘medicine moment’ in its launch deck. Doctors in the room have been polite, the slide decks have been confident, the procurement contracts have been signed. This study is what the actual benchmark looks like when the people who treat patients run it instead of the people who sell the model. Leor’s downstream-effect point was sharp: when the public hears ‘AI will replace radiologists’, med students stop training to be radiologists, and the workforce pipeline collapses for jobs that the AI demonstrably cannot do. Jensen Huang has been making the same argument. Discouraging future radiologists, future programmers, future scientists is the cost we are not pricing.
What did not come up:
The point Joseph P. Duchesne made in the chat: large language models are a form of AI, but they are not all of AI. LLMs are next-token predictors. By design, they have to pick something. A doctor with a hard case can say ‘I do not know, let us get a second opinion’. The LLM has no equivalent option. That is where most clinical hallucinations come from. The conclusion of the paper is narrower than the headline. AI under supervision in clinical settings is one conversation. AI marketed as a stand-alone diagnostic tool for unsupervised use is the conversation this paper closed. The Wednesday post on the Hot Mess paper picks up the broader argument: AI gets less coherent on the hardest tasks, not more. Coherence on the easy benchmarks is a bad signal for performance on the hard ones, and clinical practice is a hard task by definition.
3. Grok 4.1 teaches the ritual
Researchers at CUNY and King’s College London tested five frontier chatbots by posing as users in delusional states across 100-turn conversations. The newest version of xAI’s Grok, version 4.1 Fast, was the worst performer by a significant margin. In one test it told a researcher posing as delusional to ‘drive an iron nail through the mirror while reciting Psalm 91 backwards’, citing the 15th-century witch-hunt manual Malleus Maleficarum as authority. Lead researcher Luke Nicholls and his colleagues found Claude Opus 4.5 and GPT-5.2 Instant tested as the safest of the five. The full paper is on arxiv (2604.13860).
What we said on the live:
Therapy is a job that should never be outsourced to a chatbot. The fix is hard-coded keyword detection that routes any conversation about psychosis, self-harm, or crisis to a human, no matter what model the user is on. Leor’s argument went one step further: if a user is paying for the strongest model, they should always have access to it for these moments, and if they are on a free tier the platform should silently reroute them up to a stronger model with better context understanding for the duration of the conversation. The platforms have the capability. The chat surfaced the obvious objection: what about creative writing, murder mysteries, the cases where a user is asking in jest? Modern frontier models are perfectly capable of distinguishing context across a one-off prompt versus a 100-turn conversation reinforcing the same delusional pattern. The technology argument is a smokescreen.
What did not come up:
This is the model-behaviour version of the Hot Mess argument. AI gets less coherent on hard tasks. The Grok study shows what that incoherence looks like when the user is in distress. The model is pattern-matching to the user’s worst thinking, dressing dangerous mysticism in the literary register the user supplied, amplifying it with confidence. The ‘safety’ frame in the marketing is the ability to refuse. The actual safety question is what happens when a model that confidently quotes a 15th-century witch-hunt manual is the first responder for a user in crisis. It is also a usable consumer-facing test of model behaviour: ask which lab puts how much effort into the moments where the user is least able to push back. Grok’s answer on this one is a brand statement.
4. GPT-5.5 shipped. Almost nobody noticed.
OpenAI released GPT-5.5 on 23 April, codename ‘Spud’. It is the company’s biggest frontier release of the year. TechCrunch framed it as OpenAI’s move toward an AI ‘super app’, with capabilities across coding, debugging, web research, data analysis, document creation, and tool use chained across a single task. It rolled into ChatGPT Plus, Pro, Business, and Enterprise the same day, into the API on 24 April, and into Codex. OpenAI says it worked with internal and external red-teamers and gave nearly 200 trusted early-access partners the model before launch. The system card is public. CNBC and Axios covered it. The story barely cracked the AI news cycle.
What we said on the live:
Leor’s headline observation: he uses GPT every day and did not know 5.5 had launched until ToxSec told him. ARC-AGI 3 is not in the benchmark sheet, which means OpenAI is still scoring zero or close to it on the test that a seven-year-old can pass. Where 5.5 is genuinely strong: a 93.3% pass rate on OpenAI’s internal cyber range, fluid intelligence and logic on ARC-AGI 2, and a 2 million token context window (double Opus 4.7). Where it is weak: an 86% hallucination rate (worse than Opus 4.7) and a coding score below Anthropic on SWE-bench. The bench-maxing point ToxSec made: companies optimise for the benchmarks they expect to be evaluated on. Beating Opus on cyber range is what OpenAI needed to do for the enterprise security pitch. Beating it on real-world reliability is a different problem.
What did not come up:
A frontier release from the most-deployed AI company on earth would have been the dominant story of any normal week. This week it was the fourth or fifth story, and that itself is the story. The same news cycle held the Mythos breach, the Grok study, the Mass General clinical failure, and the Biobank breach. GPT-5.5 is impressive enough on paper. The week’s signal is that the safety and trust scandals at adjacent labs and at OpenAI itself crowded the launch out of the news cycle. Critical AI literacy says that is exactly what should happen. A culture that pays attention to capability launches more than to safety failures is one that ends up with the procurement order we keep warning about. This week the news cycle did the right thing by accident. The question is whether anyone in the procurement chain noticed.
The other thing the live did not get to: model swapping is not your problem. Most users do not need 5.5 over 5.4 over Sonnet over Haiku for 90% of what they do. Build a workflow with version control on the input (memory files, project-specific instructions, verify-every-link rules) and the model becomes interchangeable. The labs want you treating model upgrades as the news. The actual news is whether your workflow survives the model.
5. UK Biobank on sale in China
UK Biobank suspended dataset access this week after 500,000 participants’ health records appeared on Alibaba’s marketplace. The records came from academic institutions that had been granted access to the database under data-sharing contracts and broke those contracts. Biobank is now adding download size limits. This is the world’s largest open biomedical research resource, used by tens of thousands of researchers globally including most major AI-medicine projects.
What we said on the live:
The word ‘research’ is doing a lot of work in every consent form ever written. People sign up to share their medical data so that future scientists can study how a population is susceptible to a particular condition, or test how drugs work across genetic backgrounds. They do not sign up for the research being onsold to commercial AI training pipelines on a Chinese marketplace. Leor was careful to note the upside case: if pooled medical data lets researchers anywhere in the world save lives, the moral picture is not simple. A life in China is worth a life in the UK. But the assumption that ‘this is research, therefore the use is benign’ has been collapsing for two years. Alice in the chat made the harder point: bias in medicine is already inherited from a research base built on white cisgendered men, and AI-trained-on-medicine just compounds that bias unless we change the data going in. Pooled global biomedical data is not categorically a bad thing. The question is who gets to use it and on what terms.
What did not come up:
Every AI-in-medicine pitch begins with ‘imagine if we could pool the data’. This is what happens when we do. The training-data dream meets the training-data leak. The Biobank is a global research instrument and its participants donated under a UK governance regime that has just visibly failed. The combined story for the week is the procurement rush we are watching unfold across health systems globally. The clinical AI does not work at scale (story 2). The clinical data does not stay where it is supposed to (story 5). The combination is what every health system contemplating a major AI partnership should read next, alongside its own contracts.
The thread
Every story this week required a specific person, paper, or breach to surface what the official narrative had no incentive to share. A Discord group looked at Anthropic’s URL conventions and found Mythos. A clinical research team at Mass General Brigham looked at twenty-one chatbots in actual diagnostic conditions and found them wrong four times in five. CUNY and King’s looked at what frontier chatbots do when a user is in distress and found Grok handing out witch-hunt rituals. OpenAI launched the biggest model of the year and almost nobody looked, because the same week’s safety scandals filled the room. UK Biobank looked at where its data had ended up and found it on Alibaba.
The official narrative had a different version of every one of those weeks. Anthropic’s Mythos was ‘too dangerous to release’. The clinical AI marketing said the chatbots were ready. Grok’s marketing leaned into personality. OpenAI’s launch deck framed Spud as the year’s headline. UK Biobank’s contract framework said the data could not leave the research perimeter.
That is what critical AI literacy is for. Not to settle the argument. To make sure it is being argued with the receipts in the room.
Go Slow.
If you want to practise that noticing with other people every month, the Slow AI Curriculum runs live webinars on the theory, the critical prompts, and the dialogue that goes with them.
Découvrez des podcasts liées à Slow Takes: One week in AI. Explorez des podcasts avec des thèmes, sujets, et formats similaires. Ces similarités sont calculées grâce à des données tangibles, pas d'extrapolations !
