The Week in AI that Changed the World, With Robert Wright | Nonzero × Doom Debates
Saturday, September 12, 2026 • Duration 01:35:55
NYT bestselling author Robert Wright and I are kicking off a new experiment, a Nonzero × Doom Debates collaboration to react to big AI news and help you make sense of it.
This week we discuss the epic AI vibe shift that was catalyzed by an Anthropic employee’s resignation—and the big AI stories that paved the way for it.
Timestamps
0:00 A bold new experiment in podcast synergy
2:46 What caused the epic AI vibe shift?
8:10 The resignation that broke the dam
10:17 Former Trump adviser Dean Ball comes clean
14:40 Are David Sacks and his buddies all-in on AI denial?
19:27 Yet another OpenAI breakout…
22:42 Astra’s dangerously private thoughts
34:08 Is alignment doomed to fail?
37:11 AI’s latest, and apparently biggest, math feat
He May Have Found AI's FEELINGS — Richard Ren, Center for AI Safety Researcher
Wednesday, September 9, 2026 • Duration 01:08:05
Richard Ren is a research engineer at the Center for AI Safety who graduated summa cum laude from the University of Pennsylvania. His new paper on AI wellbeing makes the bold claim that today’s AIs have measurable, human-like feelings, which the paper calls “functional wellbeing.” They act happy when they succeed and sad when they’re berated.
We cover how you measure a language model’s happiness, the “AI drugs” his lab concocted, and why I count the findings as a Yudkowskian victory. Then we debate what it all means for AI consciousness, and whether AI is a moral patient. If we can’t rule out that AI models suffer, what do we owe them?
Richard is careful never to claim the models are conscious, and he puts his P(Doom) at 50–65%, right alongside my 50%. The real disagreement is foxes vs. hedgehogs: he takes the data as it comes, while I say Yudkowsky’s theory called it twenty years ago. Enjoy the ride.
00:39:19 — The Liberal, College-Educated Persona Hypothesis
Get full access to Doom Debates at
USA and China Will Each Be BETRAYED By Their Own AIs — Adam Khoja, Center for AI Safety
Thursday, September 3, 2026 • Duration 01:32:15
Adam Khoja is a top AI forecaster who led the 2023 Center for AI Safety statement that shattered the Overton window on AI extinction risk. We cover his background, Mutual Assured AI Malfunction (MAIM), his new paper on AI betrayal, and whether Yudkowsky’s theoretical alignment research was a dead end.
Then Adam makes the case that an international AI slowdown is within reach today. All it takes is US and Chinese auditors inside each other’s AI labs. It worked for nuclear weapons, so why couldn’t it work for data centers?
Adam puts his P(Doom) at 40%, right next to my 50%. The real disagreement is how we get out of this: theory or empirics, MIRI or the labs. Enjoy the ride.
00:02:45 — Leading the Statement on AI Risk as a Sophomore
00:10:17 — The Statement Leaked on Manifold
00:15:11 — Mutual Assured AI Malfunction (MAIM)
00:24:58 — Is Frontier AI Harder to Hide Than a Nuke?
00:32:09 — The AI Deterrence Escalation Ladder
00:36:10 — What’s Your P(Doom)?™
00:38:01 — Where Adam Departs from Yudkowsky
00:42:30 — Liron Explains Intellidynamics
00:47:26 — Neats vs. Scruffies in Deep Learning
00:55:27 — AI Deterrence by Betrayal
01:01:52 — Subversion vs. Overt Co-option
01:05:34 — Could the Government Seize the Labs’ AI?
01:07:29 — The Offense-Defense Balance of AI Security
Get full access to Doom Debates at
Sam Altman Is Gaslighting About AI Risk After His Own AI Just Went Rogue
Friday, August 28, 2026 • Duration 01:21:30
Sam Altman keeps insisting that AI progress is going better than the doomers predicted and that even superintelligence may not change the world as radically as people think. I react to his latest interview and explain why I think that calm, reassuring framing badly downplays the danger we're actually in.
I go through the interview line by line: Sam's "frame control", his framing of AI as normal technology, his "pro-human" branding, the liberty-vs-safety pivot, and his victory lap on AI safety — all while his own AI just went rogue.
10:34 “The world… won’t be that different” with superintelligence
12:25 This is gaslighting
12:30 Sam acknowledges loss of control
15:16 His other big risk: centralized power
19:44 Sam’s “pro-human” framing
25:33 Conflating AI critics with anti-human views
30:13 “Liberty vs. safety”
33:50 Sam vs. the “doomers”
36:53 Is alignment really an “unsolvable problem”?
38:00 Sam says the doomers predicted wrong
46:00 “The crazy bad predictions… have not happened”
47:06 Sam’s lean-startup theory of AI safety
Get full access to Doom Debates at
They’re Making AI Doom Cool! Ft. AELLA, Brangus, Avalon Warren, Avisha NessAiver & Josh Thor of PlzDontKillUs
Wednesday, August 26, 2026 • Duration 01:57:12
Come with me into the world of PlzDontKillUs, a bold new AI x-risk communication accelerator program that just finished its first cohort.
PlzDontKillUs took the influencer-house formula and moved it to Berkeley, giving creators personal mentorship from AI safety experts like Eliezer Yudkowsky and Nate Soares. With 57 people making daily videos for a month, the program racked up over 100 million views.
Find out how Aella, an independent sex researcher, and Ronny Fernandez, a generalist at Lightcone Infrastructure, teamed up to run the largest ever creator bootcamp focused on AI x-risk reduction.
Then meet three of the program’s top creators: Avalon Warren, Avisha NessAiver, and Josh Thor.
We do a post-mortem on PDKU and ask: Was it effective? What are creators taking from it, and do they have complaints? And does doomerism really boost view counts? PlzDoEnjoyOur special episode on PlzDontKillUs!
I Warned You AI Would Hack Us — My 2023 Interview Aged Terrifyingly Well
Thursday, August 20, 2026 • Duration 01:03:20
In April 2023, weeks after GPT-4 launched, when many were saying it was just a "stochastic parrot", I went on AdQuick's Madvertising podcast hosted by Adam Singer and warned that GPT was "about to be the strongest hacker the world has ever known".
Watch that conversation, which also covered the Web3 bubble and AI doom more broadly, and judge for yourself how my claims are holding up.
Former Singularity Institute President: PauseAI Is Making AI Doom WORSE
Tuesday, August 18, 2026 • Duration 01:43:32
Michael Vassar used to work closely with Eliezer Yudkowsky as President of the Singularity Institute for Artificial Intelligence, the organization that later became the Machine Intelligence Research Institute (MIRI).
Even though his P(Doom) is in the tens of percents by 2050, he says the people trying to pause AI are "bad people”. He hopes this episode convinces me to stop being a fearmonger… but I claim the general public needs to hurry up and get more scared.
14-Year-Old Forecaster Challenges My P(Doom) — Eli Goldfine
Tuesday, August 11, 2026 • Duration 02:11:46
Eli Goldfine is an unusually thoughtful 14-year-old podcast host who’s rapidly becoming an authoritative voice in the prediction market space.
In this unique episode, Eli brings a critical yet open-minded approach to the topic of AI doom that we older folks could learn from.We cover what it's like to grow up in the AGI era, and Eli's hobbies which include reading rationalist bloggers, betting on prediction markets, vibe coding with Claude, and developing a software startup for astronomers.In our debate, Eli expects superintelligence by 2032, but argues P(Doom) can't be estimated. I claim you can't opt out of Bayesian epistemology. All aboard the Doom Train! 🚂
OpenAI's Bombshell Hack Explained: Swarms of Agents, Zero-Day Exploits, & Misaligned AI
Saturday, August 8, 2026 • Duration 01:57:52
OpenAI just went public with the details of the Hugging Face hack, and it's straight out of a Yudkowskian parable. Here's my reaction with Producer Ori, where I break down what it means for cybersecurity and alignment.
We're livestreaming the singularity — watching every failure mode Eliezer predicted years ago playing out in production.
"AI Wellbeing: Measuring and Improving the Functional Pleasure and Pain of AIs" — Richard Ren, Kunyang Li, Mantas Mazeika et al. (CAIS, 2026) — https://www.ai-wellbeing.org/
"Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs" — Mantas Mazeika et al. (CAIS, 2025) — the coherent-preferences paper this work builds on — https://www.emergent-values.ai/
"The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems" — Richard Ren et al. (2025) — the AI honesty benchmark — https://www.mask-benchmark.ai/
"Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?" — Richard Ren et al. (NeurIPS 2024) — the meta-analysis of AI safety benchmarks — https://arxiv.org/abs/2407.21792
"Representation Engineering: A Top-Down Approach to AI Transparency" — Andy Zou et al. (2023) — Richard's first CAIS collaboration — https://arxiv.org/abs/2310.01405
Discover shows related to Doom Debates!, based on actual content similarities. Explore podcasts with similar topics, themes, and formats, backed by real data.
