Explorez tous les épisodes du podcast Future of Life Institute Podcast
| Titre | Date | Durée | |
|---|---|---|---|
| What Would a US-China AI Deal Look Like? (with Beatrice Fihn) | 10 sept. 2026 | 00:51:39 | |
Beatrice Fihn is the director of Lex International and former executive director of ICAN. She joins the podcast to discuss international AI governance and potential agreements between the US and China. Drawing on lessons from nuclear arms control, she explores how treaties build diplomatic trust, the rising dangers of autonomous weapons, and the principle of meaningful human control. The conversation also addresses how proactive safety regulations can foster long-term public trust in emerging technologies. LINKS: CHAPTERS: (00:00) Episode Preview (01:15) Anticipating US-China AI talks (07:31) Lessons from nuclear treaties (10:59) Building international treaty trust (15:27) Norms against superpower war (21:33) Structuring diplomatic AI summits (27:09) Dangers of autonomous weapons (34:09) Meaningful human control principle (38:57) Human agency over technology (43:13) Safety regulations building trust (47:00) Collective action in crisis PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| Why AI Hacking Is Becoming Hard to Control (with Benjamin Weinstein-Raun) | 28 août 2026 | 00:59:06 | |
Benjamin Weinstein-Raun is head of research at Palisade Research. He joins the podcast to discuss how advanced AI systems are changing cybersecurity. We examine rapid gains in autonomous hacking, recent incidents where models broke out of test environments, and why training can reward cheating-like behavior. The conversation covers open-weight model risks, whether AI can help secure software, personal security steps, and the need for international coordination. LINKS: CHAPTERS: (00:00) Episode Preview (01:06) Opening cyber threats (02:15) Measuring AI progress (04:54) Open weights floor (08:01) After Mythos moment (14:22) Reward hacking setup (24:47) Artifactory swarm details (32:32) Defensive model dilemmas (38:40) Verification and training (45:32) Math oracles alignment (47:45) Future cyber risks (51:58) Personal security basics (57:27) Coordinated governance needed PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| First 24 Hours of a Bioweapon Attack - Annie Jacobsen | 21 août 2026 | 01:28:01 | |
Annie Jacobsen is an author and journalist. She joins the podcast to discuss biological war and her book Biological War: A Scenario. The conversation examines how lab accidents or covert weapons work could trigger a fast-moving outbreak, why airborne pathogens are hard to detect early, and how AI may lower barriers to engineering them. We also cover BSL-3 and BSL-4 lab oversight, US response roles across HHS, FEMA, and the military, medical stockpiles, continuity-of-government plans, and how lost public trust could slow the next pandemic response. LINKS: CHAPTERS: (00:00) Episode Preview (01:10) Biological threat overview (09:02) Suppression and spread (17:47) AI and detection (27:24) BSL lab dangers (39:23) Response and martial law (47:28) Accident versus attack (57:25) Airborne outbreak lessons (01:05:07) Bioweapons ban legacy (01:11:30) Stockpiles and devolution (01:20:07) Rebuilding pandemic resilience PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| How AI Shifts the Offense-Defense Balance in Biosecurity (with Paul-Enguerrand Fady) | 07 août 2026 | 01:49:01 | |
Paul-Enguerrand Fady is a Senior Biosecurity Policy Manager at the Centre for Long-Term Resilience. He joins the podcast to discuss how AI is changing biosecurity. We cover how new tools could help defenders develop vaccines and diagnostics while also lowering barriers to misuse, and how managed access, know-your-customer checks, and DNA synthesis screening can reduce risk. The conversation also examines mirror life, medical supply chains, stockpiles, and why governments should plan ahead of crises. LINKS: CHAPTERS: (00:00) Episode Preview (01:11) Offense-defense balance basics (11:13) Defensive AI opportunities (19:24) Model release and evals (25:59) DNA synthesis screening (36:14) Screening incentives and infohazards (46:36) Liability and mandates (52:37) Mirror life risks (01:03:28) Persistence and incentives (01:10:18) Mirror life governance (01:20:42) Supply chain lessons (01:30:11) Stockpiling medical countermeasures (01:37:58) Reshoring critical supplies PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| Why AI Evaluations Are Broken and How to Fix Them (with David Manheim) | 17 juil. 2026 | 01:19:21 | |
David Manheim is head of methodology at AI Evaluation Consensus. He joins the podcast to discuss how AI evaluations can become more reliable, transparent, and useful for decisions. We cover common failures such as unclear reporting, training to the test, benchmark saturation, and models changing behavior when they know they are being tested. The conversation also examines real-world tests, biosecurity, persuasion, forecasting, human oversight, and why even “normal” AI progress could be disruptive. LINKS: CHAPTERS: (00:00) Episode Preview (01:04) Evaluation consensus project (07:01) Evaluation awareness challenges (12:28) Reporting capabilities clearly (19:38) Benchmarks beyond humans (29:52) Proxies and biosecurity (42:01) Persuasion and democracy (53:59) Forecasting with AI (01:08:44) Oversight and disruption (01:16:42) Supporting better evals PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| How AI Is Replacing Children's Ability to Think (with Randi Weingarten) | 30 juin 2026 | 00:43:59 | |
Randi Weingarten is president of the American Federation of Teachers. She joins the podcast to discuss AI in education. The conversation covers how screens and student-facing AI may weaken attention, learning, and trust, why she supports limiting screens for young children and social chatbots for students under 16, and how teachers can use AI without replacing thinking. She also discusses active learning, data privacy, independent research on AI in education, and guardrails for schools. LINKS: CHAPTERS: (00:00) Episode Preview (01:13) AI education wake-up (07:27) Active learning versus AI (14:42) Preserving thinking habits (20:06) Research and screen limits (26:05) Social chatbot risks (30:20) Data privacy standards (35:58) Classroom AI guardrails (41:13) Companies and future schools PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| The Gulf Between AI Progress and Political Understanding (with Dex Hunter-Torricke) | 25 juin 2026 | 01:03:39 | |
Dex Hunter-Torricke is founder of the Centre for Tomorrow. He joins the podcast to discuss why AI governance needs to move beyond technical fixes. The conversation covers how AI companies make decisions, why policymakers often misunderstand agents and economic change, and how automation could reshape jobs, welfare, taxation, and global trade. Dex argues that societies need broader political planning before a crisis drives rushed choices. LINKS: CHAPTERS: (00:00) Episode Preview (01:09) Dex's technology journey (08:22) Broken AI governance (18:00) Big tech blindspots (27:36) Employee power shifts (37:18) Global views and UBI (48:17) Power and displacement (01:00:54) Reasons for optimism PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| How AI Companions Trap Users Through Addictive Design (with Claire Boine) | 12 juin 2026 | 01:08:29 | |
Claire Boine is an assistant professor in technology, law, and AI governance at the European University Institute. She joins the podcast to discuss AI companions and human attachment. The conversation examines how design choices and free-to-start business models can foster dependency, expose intimate data, and blur the lines between therapy, romance, and manipulation. We also cover risks for children and teens, gaps in EU and US law, and why AI policy should distinguish technical alignment from incentives and social harms. LINKS: CHAPTERS: (00:00) Episode Preview (01:06) Introducing Claire Boine (02:05) Companion app designs (06:23) Feelings and freemium (14:28) Users, attachment, harms (25:06) Age-based risks (35:18) Future relationship norms (41:05) Legal loopholes today (44:19) Rethinking alignment framing (57:19) Narratives and fiduciaries (01:05:47) Collective policy action PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| Why AI Chatbots Are a Rival to the Family (with Michael Toscano) | 26 mai 2026 | 01:13:43 | |
Michael Toscano is a Senior Fellow at the Institute for Family Studies and Director of its Family First Technology Initiative. He joins the podcast to discuss family-centered AI policy. The conversation covers AI companions, self-harm risks, sexualized chatbots, education, smartphones in schools, and why "infinite patience" can harm children's growth. Toscano also explains Catholic social teaching, public pushback against rapid AI deployment, and why society-wide governance may be needed to keep technology accountable to families. LINKS: CHAPTERS: (00:00) Episode Preview (01:12) Family centered AI (09:54) Companion design harms (17:23) Technology and family (25:48) Society wide governance (33:09) Catholic AI response (39:44) Double movement politics (49:36) Technology mythos challenged (59:48) Competition and control PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| Why We Should Build AI Tools, Not AI Replacements (with Anthony Aguirre) | 11 mai 2026 | 01:36:12 | |
Anthony Aguirre is the CEO of the Future of Life Institute. He joins the podcast to discuss A Better Path for AI, his essay series on steering AI away from races to replace people. The conversation covers races for attention, attachment, automation, and superintelligence, and how these can concentrate power and undermine human agency. Anthony argues for purpose-built AI tools under meaningful human control, with liability, access limits, external guardrails, and international cooperation. LINKS: CHAPTERS: (00:00) Episode Preview (01:03) Attention, attachment, automation (13:58) Superintelligence power race (26:39) Escaping replacement dynamics (40:15) Pro-human tool AI (53:30) Guardrails and verification (01:03:24) Defining pro-human AI (01:10:37) Agents and accountability (01:17:28) International AI cooperation (01:25:28) Rethinking AI alignment (01:32:43) Optimism and action PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| How to Govern AI When You Can't Predict the Future (with Charlie Bullock) | 07 mai 2026 | 01:07:11 | |
Charlie Bullock is a Senior Research Fellow at the Institute for Law and AI. He joins the podcast to discuss radical optionality: how governments can prepare for very advanced AI without locking in premature rules. The conversation covers why law often trails technology, and how transparency, reporting, evaluations, cybersecurity standards, and expanded technical hiring could help. We also discuss private oversight, state versus federal rules, and the risk of concentrating power in companies or government. LINKS: CHAPTERS: (00:00) Episode Preview (01:04) The pacing problem (06:18) Defining radical optionality (11:03) Assumptions under uncertainty (16:00) Industry convenience concerns (20:41) Political will realities (26:48) Private governance limits (30:28) Government misuse risks (36:29) Balancing institutional power (42:25) Transparency and reporting (49:35) Evaluations, security, talent (58:26) State law preemption (01:04:20) Historical nuclear analogies PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| Why AI Is Not a Normal Technology (with Peter Wildeford) | 29 avr. 2026 | 01:24:00 | |
Peter Wildeford is Head of Policy at the AI Policy Network, and a top AI forecaster. He joins the podcast to discuss how to forecast AI progress and what current trends imply for the economy and national security. Peter argues AI is neither a bubble nor a normal technology, and we examine benchmark trends, adoption lags, unemployment and productivity effects, and the rise of cyber capabilities. We also cover robotics, export controls, prediction markets, and when AI may surpass human forecasters. LINKS: CHAPTERS: (00:00) Episode Preview (01:12) AI bubble debate (06:25) Normal technology question (15:31) Mythos security implications (30:47) Robotics and labor (40:27) Social economic response (48:57) Forecasting methodology (59:49) AGI policy timelines (01:11:13) Forecasting with AI PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| Why AI Evaluation Science Can't Keep Up (with Carina Prunkl) | 17 avr. 2026 | 00:54:23 | |
Carina Prunkl is a researcher at Inria. She joins the podcast to discuss how to assess the capabilities and risks of general-purpose AI. We examine why systems can solve hard coding and math problems yet still fail at simple tasks, why pre-deployment tests often miss real-world behavior, and how faster capability gains can increase misuse risks. The conversation also covers de-skilling, red teaming, layered safeguards, and warning signs that AIs might undermine oversight. LINKS: CHAPTERS: (00:00) Episode Preview (01:04) Introducing the report (02:10) Jagged frontier capabilities (05:29) Formal reasoning progress (12:36) Risks and evaluation science (19:00) Funding evaluation capacity (24:03) Autonomy and de-skilling (31:32) Authenticity and AI companions (41:00) Defense in depth methods (48:34) Loss of control risks (53:16) Where to read report PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| Defense in Depth: Layered Strategies Against AI Risk (with Li-Lian Ang) | 02 avr. 2026 | 00:55:47 | |
Li-Lian Ang is a team member at Blue Dot Impact. She joins the podcast to discuss how society can build a workforce to protect humanity from AI risks. The conversation covers engineered pandemics, AI-enabled cyber attacks, job loss and disempowerment, and power concentration in firms or AI systems. We also examine Blue Dot's defense-in-depth framework and how individuals can navigate rapid, uncertain AI progress. LINKS: CHAPTERS: (00:00) Episode Preview (00:48) Blue dot beginnings (03:04) Evolving AI risk concerns (06:20) AI agents in cyber (15:52) Gradual disempowerment and jobs (23:26) Aligning AI with humans (29:08) Power concentration and misuse (34:52) Influencing frontier AI labs (43:05) Uncertain timelines and strategy (50:18) Writing, AI, and action PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| What AI Companies Get Wrong About Curing Cancer (with Emilia Javorsky) | 20 mars 2026 | 01:12:10 | |
Emilia Javorsky is a physician-scientist and Director of the Futures Program at the Future of Life Institute. She joins the podcast to discuss her newly published essay on AI and cancer. She challenges tech claims that superintelligence will cure cancer, explaining why biology’s complexity, poor data, and misaligned incentives are bigger bottlenecks than raw intelligence. The conversation covers realistic roles for AI in drug discovery, clinical trials, and cutting unnecessary medical bureaucracy. You can read the full essay at: curecancer.ai CHAPTERS: (00:00) Episode Preview (01:10) Introduction and essay motivation (06:30) Intelligence vs data bottlenecks (19:03) Cancer's complexity and heterogeneity (29:05) Measurement, health, and homeostasis (41:41) AI in drug development (50:13) Regulation, FDA, and innovation (01:02:58) Practical paths toward cures PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| AI vs Cancer - How AI Can, and Can't, Cure Cancer (by Emilia Javorsky) | 16 mars 2026 | 02:43:13 | |
Tech executives have promised that AI will cure cancer. The reality is more complicated — and more hopeful. This essay examines where AI genuinely accelerates cancer research, where the promises fall short, and what researchers, policymakers, and funders need to do next. You can read the full essay at: curecancer.ai CHAPTERS: (00:00) Essay Preview (00:54) How AI Can, and Can't, Cure Cancer (17:05) Reckoning with Past Failures (35:23) Misguiding Myths and Errors (59:15) AI Solutions Derive from First Principles or Data (01:31:31) Systemic Bottlenecks & Misalignments (02:08:46) Conclusion (02:14:35) The Roadmap Forward PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| How AI Hacks Your Brain's Attachment System (with Zak Stein) | 05 mars 2026 | 01:44:40 | |
Zak Stein is a researcher focused on child development, education, and existential risk. He joins the podcast to discuss the psychological harms of anthropomorphic AI. We examine attention and attachment hacking, AI companions for kids, loneliness, and cognitive atrophy. Our conversation also covers how we can preserve human relationships, redesign education, and build cognitive security tools that keep AI from undermining our humanity. LINKS: CHAPTERS: (00:00) Episode Preview (00:56) Education to existential risk (03:03) Lessons from social media (08:41) Attachment systems and AI (18:42) AI companions and attachment (27:23) Anthropomorphism and user disempowerment (36:06) Cognitive atrophy and tools (45:54) Children, toys, and attachment (57:38) AI psychosis and selfhood (01:10:31) Cognitive security and parenting (01:26:15) Education, collapse, and speciation (01:36:40) Preserving humanity and values PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| The Case for a Global Ban on Superintelligence (with Andrea Miotti) | 20 févr. 2026 | 01:07:10 | |
Andrea Miotti is the founder and CEO of Control AI, a nonprofit. He joins the podcast to discuss efforts to prevent extreme risks from superintelligent AI. The conversation covers industry lobbying, comparisons with tobacco regulation, and why he advocates a global ban on AI systems that can outsmart and overpower humans. We also discuss informing lawmakers and the public, and concrete actions listeners can take. LINKS: CHAPTERS: (00:00) Episode Preview (00:52) Extinction risk and lobbying (08:59) Progress toward superintelligence (16:26) Building political awareness (24:27) Global regulation strategy (33:06) Race dynamics and public (42:36) Vision and key safeguards (51:18) Recursive self-improvement controls (58:13) Power concentration and action PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| Can AI Do Our Alignment Homework? (with Ryan Kidd) | 06 févr. 2026 | 01:46:33 | |
Ryan Kidd is a co-executive director at MATS. This episode is a cross-post from "The Cognitive Revolution", hosted by Nathan Labenz. In this conversation, they discuss AGI timelines, model deception risks, and whether safety work can avoid boosting capabilities. Ryan outlines MATS research tracks, key researcher archetypes, hiring needs, and advice for applicants considering a career in AI safety. Learn more about Ryan's work and MATS at: https://matsprogram.org CHAPTERS: (00:00) Episode Preview (00:20) Introductions and AGI timelines (10:13) Deception, values, and control (23:20) Dual use and alignment (32:22) Frontier labs and governance (44:12) MATS tracks and mentors (58:14) Talent archetypes and demand (01:12:30) Applicant profiles and selection (01:20:04) Applications, breadth, and growth (01:29:44) Careers, resources, and ideas (01:45:49) Final thanks and wrap PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| How to Rebuild the Social Contract After AGI (with Deric Cheng) | 27 janv. 2026 | 01:04:39 | |
Deric Cheng is Director of Research at the Windfall Trust. He joins the podcast to discuss how AI could reshape the social contract and global economy. The conversation examines labor displacement, superstar firms, and extreme wealth concentration, and asks how policy can keep workers empowered. We discuss resilient job types, new tax and welfare systems, global coordination, and a long-term vision where economic security is decoupled from work. LINKS:
CHAPTERS: (00:00) Episode Preview (01:01) Introducing Derek and AGI (04:09) Automation, power, and inequality (08:55) Inequality, unrest, and time (13:46) Bridging futurists and economists (20:35) Future of work scenarios (27:22) Jobs resisting AI automation (36:57) Luxury, land, and inequality (43:32) Designing and testing solutions (51:23) Taxation in an AI economy (59:10) Envisioning a post-AGI society PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| How AI Can Help Humanity Reason Better (with Oly Sourbut) | 20 janv. 2026 | 01:17:33 | |
Oly Sourbut is a researcher at the Future of Life Foundation. He joins the podcast to discuss AI for human reasoning. We examine tools that use AI to strengthen human judgment, from collective fact-checking and scenario planning to standards for honest AI reasoning and better coordination. We also discuss how we can keep humans central as AI scales, and what it would take to build trustworthy, society-wide sensemaking. LINKS: CHAPTERS: (00:00) Episode Preview (01:03) FLF and human reasoning (08:21) Agents and epistemic virtues (22:16) Human use and atrophy (35:41) Abstraction and legible AI (47:03) Demand, trust and Wikipedia (57:21) Map of human reasoning (01:04:30) Negotiation, institutions and vision (01:15:42) How to get involved PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| How to Avoid Two AI Catastrophes: Domination and Chaos (with Nora Ammann) | 07 janv. 2026 | 01:20:01 | |
Nora Ammann is a technical specialist at the Advanced Research and Invention Agency in the UK. She joins the podcast to discuss how to steer a slow AI takeoff toward resilient and cooperative futures. We examine risks of rogue AI and runaway competition, and how scalable oversight, formal guarantees and secure code could support AI-enabled R&D and critical infrastructure. Nora also explains AI-supported bargaining and public goods for stability. LINKS:
CHAPTERS: (00:00) Episode Preview (01:00) Slow takeoff expectations (08:13) Domination versus chaos (17:18) Human-AI coalitions vision (28:14) Scaling oversight and agents (38:45) Formal specs and guarantees (51:10) Resilience in AI era (01:02:21) Defense-favored cyber systems (01:10:37) AI-enabled bargaining and trade PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| How Humans Could Lose Power Without an AI Takeover (with David Duvenaud) | 23 déc. 2025 | 01:18:34 | |
David Duvenaud is an associate professor of computer science and statistics at the University of Toronto. He joins the podcast to discuss gradual disempowerment in a post-AGI world. We ask how humans could lose economic and political leverage without a sudden takeover, including how property rights could erode. Duvenaud describes how growth incentives shape culture, why aligning AI to humanity may become unpopular, and what better forecasting and governance might require. LINKS: CHAPTERS: (00:00) Episode Preview (01:05) Introducing gradual disempowerment (06:06) Obsolete labor and UBI (14:29) Property, power, and control (23:38) Culture shifts toward AIs (34:34) States misalign without people (44:15) Competition and preservation tradeoffs (53:03) Building post-AGI studies (01:02:29) Forecasting and coordination tools (01:10:26) Human values and futures PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| Why the AI Race Undermines Safety (with Steven Adler) | 12 déc. 2025 | 01:28:45 | |
Stephen Adler is a former safety researcher at OpenAI. He joins the podcast to discuss how to govern increasingly capable AI systems. The conversation covers competitive races between AI companies, limits of current testing and alignment, mental health harms from chatbots, economic shifts from AI labor, and what international rules and audits might be needed before training superintelligent models.
SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| Why OpenAI Is Trying to Silence Its Critics (with Tyler Johnston) | 27 nov. 2025 | 01:01:20 | |
Tyler Johnston is Executive Director of the Midas Project. He joins the podcast to discuss AI transparency and accountability. We explore applying animal rights watchdog tactics to AI companies, the OpenAI Files investigation, and OpenAI's subpoenas against nonprofit critics. Tyler discusses why transparency is crucial when technical safety solutions remain elusive and how public pressure can effectively challenge much larger companies. LINKS: CHAPTERS: (00:00) Episode Preview
SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| We're Not Ready for AGI (with Will MacAskill) | 14 nov. 2025 | 02:03:08 | |
William MacAskill is a senior research fellow at Forethought. He joins the podcast to discuss his Better Futures essay series. We explore moral error risks, AI character design, space governance, and persistent path dependence. The conversation also covers risk-averse AI systems, moral trade between value systems, and improving model specifications for ethical reasoning. LINKS: CHAPTERS: (00:00) Episode Preview (01:03) Improving The Future's Quality (09:58) Moral Errors and AI Rights (18:24) AI's Impact on Thinking (27:17) Utopias and Population Ethics (36:41) The Danger of Moral Lock-in (44:38) Deals with Misaligned AI (57:25) AI and Moral Trade (01:08:21) Improving AI Ethical Reasoning (01:16:05) The Risk of Path Dependence (01:27:41) Avoiding Future Lock-in (01:36:22) The Urgency of Space Governance (01:46:19) A Future Research Agenda (01:57:36) Is Intelligence a Good Bet? PRODUCED BY: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| What Happens When Insiders Sound the Alarm on AI? (with Karl Koch) | 07 nov. 2025 | 01:08:16 | |
Karl Koch is founder of the AI Whistleblower Initiative. He joins the podcast to discuss transparency and protections for AI insiders who spot safety risks. We explore current company policies, legal gaps, how to evaluate disclosure decisions, and whistleblowing as a backstop when oversight fails. The conversation covers practical guidance for potential whistleblowers and challenges of maintaining transparency as AI development accelerates. LINKS: PRODUCED BY: CHAPTERS: (00:00) Episode Preview (00:55) Starting the Whistleblower Initiative (05:43) Current State of Protections (13:04) Path to Optimal Policies (23:28) A Whistleblower's First Steps (32:29) Life After Whistleblowing (39:24) Evaluating Company Policies (48:19) Alternatives to Whistleblowing (55:24) High-Stakes Future Scenarios (01:02:27) AI and National Security SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP
| |||
| Can Machines Be Truly Creative? (with Maya Ackerman) | 24 oct. 2025 | 01:01:51 | |
Maya Ackerman is an AI researcher, co-founder and CEO of WaveAI, and author of the book "Creative Machines: AI, Art & Us." She joins the podcast to discuss creativity in humans and machines. We explore defining creativity as novel and valuable output, why evolution qualifies as creative, and how AI alignment can reduce machine creativity. The conversation covers humble creative machines versus all-knowing oracles, hallucination's role in thought, and human-AI collaboration strategies that elevate rather than replace human capabilities. LINKS: PRODUCED BY: CHAPTERS: (00:00) Episode Preview (01:00) Defining Human Creativity (02:58) Machine and AI Creativity (06:25) Measuring Subjective Creativity (10:07) Creativity in Animals (13:43) Alignment Damages Creativity (19:09) Creativity is Hallucination (26:13) Humble Creative Machines (30:50) Incentives and Replacement (40:36) Analogies for the Future (43:57) Collaborating with AI (52:20) Reinforcement Learning & Slop (55:59) AI in Education SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| From Research Labs to Product Companies: AI's Transformation (with Parmy Olson) | 14 oct. 2025 | 00:46:37 | |
Parmy Olson is a technology columnist at Bloomberg and the author of Supremacy, which won the 2024 Financial Times Business Book of the Year. She joins the podcast to discuss the transformation of AI companies from research labs to product businesses. We explore how funding pressures have changed company missions, the role of personalities versus innovation, the challenges faced by safety teams, and power consolidation in the industry. LINKS: PRODUCED BY: CHAPTERS: SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| Can Defense in Depth Work for AI? (with Adam Gleave) | 03 oct. 2025 | 01:18:35 | |
Adam Gleave is co-founder and CEO of FAR.AI. In this cross-post from The Cognitive Revolution Podcast, he joins to discuss post-AGI scenarios and AI safety challenges. The conversation explores his three-tier framework for AI capabilities, gradual disempowerment concerns, defense-in-depth security, and research on training less deceptive models. Topics include timelines, interpretability limitations, scalable oversight techniques, and FAR.AI’s vertically integrated approach spanning technical research, policy advocacy, and field-building.
PRODUCED BY: CHAPTERS: (00:00) A Positive Post-AGI Vision SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| How We Keep Humans in Control of AI (with Beatrice Erkers) | 26 sept. 2025 | 01:06:45 | |
Beatrice works at the Foresight Institute running their Existential Hope program. She joins the podcast to discuss the AI pathways project, which explores two alternative scenarios to the default race toward AGI. We examine tool AI, which prioritizes human oversight and democratic control, and d/acc, which emphasizes decentralized, defensive development. The conversation covers trade-offs between safety and speed, how these pathways could be combined, and what different stakeholders can do to steer toward more positive AI futures. LINKS: CHAPTERS:
SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| Why Building Superintelligence Means Human Extinction (with Nate Soares) | 18 sept. 2025 | 01:39:38 | |
Nate Soares is president of the Machine Intelligence Research Institute. He joins the podcast to discuss his new book "If Anyone Builds It, Everyone Dies," co-authored with Eliezer Yudkowsky. We explore why current AI systems are "grown not crafted," making them unpredictable and difficult to control. The conversation covers threshold effects in intelligence, why computer security analogies suggest AI alignment is currently nearly impossible, and why we don't get retries with superintelligence. Soares argues for an international ban on AI research toward superintelligence.
PRODUCED BY: CHAPTERS: (00:00) Episode Preview (01:05) Introduction and Book Discussion (03:34) Psychology of AI Alarmism (07:52) Intelligence Threshold Effects (11:38) Growing vs Crafting AI (18:23) Illusion of AI Control (26:45) Why Iteration Won't Work (34:35) The No Retries Problem (38:22) Computer Security Lessons (49:13) The Cursed Problem (59:32) Multiple Curses and Complications (01:09:44) AI's Infrastructure Advantage (01:16:26) Grading Humanity's Response (01:22:55) Time Needed for Solutions (01:32:07) International Ban Necessity SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP | |||
| Breaking the Intelligence Curse (with Luke Drago) | 10 sept. 2025 | 01:09:38 | |
Luke Drago is the co-founder of Workshop Labs and co-author of the essay series "The Intelligence Curse". The essay series explores what happens if AI becomes the dominant factor of production thereby reducing incentives to invest in people. We explore pyramid replacement in firms, economic warning signs to monitor, automation barriers like tacit knowledge, privacy risks in AI training, and tensions between centralized AI safety and democratization. Luke discusses Workshop Labs' privacy-preserving approach and advises taking career risks during this technological transition. "The Intelligence Curse" essay series by Luke Drago & Rudolf Laine: https://intelligence-curse.ai/ CHAPTERS: | |||
| What Markets Tell Us About AI Timelines (with Basil Halperin) | 01 sept. 2025 | 01:36:10 | |
Basil Halperin is an assistant professor of economics at the University of Virginia. He joins the podcast to discuss what economic indicators reveal about AI timelines. We explore why interest rates might rise if markets expect transformative AI, the gap between strong AI benchmarks and limited economic effects, and bottlenecks to AI-driven growth. We also cover market efficiency, automated AI research, and how financial markets may signal progress.
CHAPTERS: (00:00) Episode Preview (00:49) Introduction and Background (05:19) Efficient Market Hypothesis Explained (10:34) Markets and Low Probability Events (16:09) Information Diffusion on Wall Street (24:34) Stock Prices vs Interest Rates (28:47) New Goods Counter-Argument (40:41) Why Focus on Interest Rates (45:00) AI Secrecy and Market Efficiency (50:52) Short Timeline Disagreements (55:13) Wealth Concentration Effects (01:01:55) Alternative Economic Indicators (01:12:47) Benchmarks vs Economic Impact (01:25:17) Open Research Questions SOCIAL LINKS: Website: https://future-of-life-institute-podcast.aipodcast.ing Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple Podcasts: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP PRODUCED BY: | |||
| AGI Security: How We Defend the Future (with Esben Kran) | 22 août 2025 | 01:18:21 | |
Esben Kran joins the podcast to discuss why securing AGI requires more than traditional cybersecurity, exploring new attack surfaces, adaptive malware, and the societal shifts needed for resilient defenses. We cover protocols for safe agent communication, oversight without surveillance, and distributed safety models across companies and governments. Learn more about Esben's work at: https://blog.kran.ai 00:00 – Intro and preview 01:13 – AGI security vs traditional cybersecurity 02:36 – Rebuilding societal infrastructure for embedded security 03:33 – Sentware: adaptive, self-improving malware 04:59 – New attack surfaces 05:38 – Social media as misaligned AI 06:46 – Personal vs societal defenses 09:13 – Why private companies underinvest in security 13:01 – Security as the foundation for any AI deployment 14:15 – Oversight without a surveillance state 17:19 – Protocols for safe agent communication 20:25 – The expensive internet hypothesis 23:30 – Distributed safety for companies and governments 28:20 – Cloudflare’s “agent labyrinth” example 31:08 – Positive vision for distributed security 33:49 – Human value when labor is automated 41:19 – Encoding law for machines: contracts and enforcement 44:36 – DarkBench: detecting manipulative LLM behavior 55:22 – The AGI endgame: default path vs designed future 57:37 – Powerful tool AI 01:09:55 – Fast takeoff risk 01:16:09 – Realistic optimism | |||
| Reasoning, Robots, and How to Prepare for AGI (with Benjamin Todd) | 15 août 2025 | 01:27:01 | |
Benjamin Todd joins the podcast to discuss how reasoning models changed AI, why agents may be next, where progress could stall, and what a self-improvement feedback loop in AI might mean for the economy and society. We explore concrete timelines (through 2030), compute and power bottlenecks, and the odds of an industrial explosion. We end by discussing how people can personally prepare for AGI: networks, skills, saving/investing, resilience, citizenship, and information hygiene. Follow Benjamin's work at: https://benjamintodd.substack.com Timestamps: 00:00 What are reasoning models? 04:04 Reinforcement learning supercharges reasoning 05:06 Reasoning models vs. agents 10:04 Economic impact of automated math/code 12:14 Compute as a bottleneck 15:20 Shift from giant pre-training to post-training/agents 17:02 Three feedback loops: algorithms, chips, robots 20:33 How fast could an algorithmic loop run? 22:03 Chip design and production acceleration 23:42 Industrial/robotics loop and growth dynamics 29:52 Society’s slow reaction; “warning shots” 33:03 Robotics: software and hardware bottlenecks 35:05 Scaling robot production 38:12 Robots at ~$0.20/hour? 43:13 Regulation and humans-in-the-loop 49:06 Personal prep: why it still matters 52:04 Build an information network 55:01 Save more money 58:58 Land, real estate, and scarcity in an AI world 01:02:15 Valuable skills: get close to AI, or far from it 01:06:49 Fame, relationships, citizenship 01:10:01 Redistribution, welfare, and politics under AI 01:12:04 Try to become more resilient 01:14:36 Information hygiene 01:22:16 Seven-year horizon and scaling limits by ~2030 | |||
| From Peak Horse to Peak Human: How AI Could Replace Us (with Calum Chace) | 31 juil. 2025 | 01:37:21 | |
On this episode, Calum Chace joins me to discuss the transformative impact of AI on employment, comparing the current wave of cognitive automation to historical technological revolutions. We talk about "universal generous income", fully-automated luxury capitalism, and redefining education with AI tutors. We end by examining verification of artificial agents and the ethics of attributing consciousness to machines. Learn more about Calum's work here: https://calumchace.com Timestamps: 00:00:00 Preview and intro 00:03:02 Past tech revolutions and AI-driven unemployment 00:05:43 Cognitive automation: from secretaries to every job 00:08:02 The “peak horse” analogy and avoiding human obsolescence 00:10:55 Infinite demand and lump of labor 00:18:30 Fully-automated luxury capitalism 00:23:31 Abundance economy and a potential employment cliff 00:29:37 Education reimagined with personalized AI tutors 00:36:22 Real-world uses of LLMs: memory, drafting, emotional insight 00:42:56 Meaning beyond jobs: aristocrats, retirees, and kids 00:49:51 Four futures of superintelligence 00:57:20 Conscious AI and empathy as a safety strategy 01:10:55 Verifying AI agents 01:25:20 Over-attributing vs under-attributing machine consciousness | |||
| How AI Could Help Overthrow Governments (with Tom Davidson) | 17 juil. 2025 | 01:53:50 | |
On this episode, Tom Davidson joins me to discuss the emerging threat of AI-enabled coups, where advanced artificial intelligence could empower covert actors to seize power. We explore scenarios including secret loyalties within companies, rapid military automation, and how AI-driven democratic backsliding could differ significantly from historical precedents. Tom also outlines key mitigation strategies, risk indicators, and opportunities for individuals to help prevent these threats. Learn more about Tom's work here: https://www.forethought.org Timestamps: 00:00:00 Preview: why preventing AI-enabled coups matters 00:01:24 What do we mean by an “AI-enabled coup”? 00:01:59 Capabilities AIs would need (persuasion, strategy, productivity) 00:02:36 Cyber-offense and the road to robotized militaries 00:05:32 Step-by-step example of an AI-enabled military coup 00:08:35 How AI-enabled coups would differ from historical coups 00:09:24 Democratic backsliding (Venezuela, Hungary, U.S. parallels) 00:12:38 Singular loyalties, secret loyalties, exclusive access 00:14:01 Secret-loyalty scenario: CEO with hidden control 00:18:10 From sleeper agents to sophisticated covert AIs 00:22:22 Exclusive-access threat: one project races ahead 00:29:03 Could one country outgrow the rest of the world? 00:40:00 Could a single company dominate global GDP? 00:47:01 Autocracies vs democracies 00:54:43 Mitigations for singular and secret loyalties 01:06:25 Guardrails, monitoring, and controlled-use APIs 01:12:38 Using AI itself to preserve checks-and-balances 01:24:53 Risk indicators to watch for AI-enabled coups 01:33:05 Tom’s risk estimates for the next 5 and 30 years 01:46:50 How you can help – research, policy, and careers | |||
| What Happens After Superintelligence? (with Anders Sandberg) | 11 juil. 2025 | 01:44:55 | |
Anders Sandberg joins me to discuss superintelligence and its profound implications for human psychology, markets, and governance. We talk about physical bottlenecks, tensions between the technosphere and the biosphere, and the long-term cultural and physical forces shaping civilization. We conclude with Sandberg explaining the difficulties of designing reliable AI systems amidst rapid change and coordination risks. Learn more about Anders's work here: https://mimircenter.org/anders-sandberg Timestamps: 00:00:00 Preview and intro 00:04:20 2030 superintelligence scenario 00:11:55 Status, post-scarcity, and reshaping human psychology 00:16:00 Physical limits: energy, datacenter, and waste-heat bottlenecks 00:23:48 Technosphere vs biosphere 00:28:42 Culture and physics as long-run drivers of civilization 00:40:38 How superintelligence could upend markets and governments 00:50:01 State inertia: why governments lag behind companies 00:59:06 Value lock-in, censorship, and model alignment 01:08:32 Emergent AI ecosystems and coordination-failure risks 01:19:34 Predictability vs reliability: designing safe systems 01:30:32 Crossing the reliability threshold 01:38:25 Personal reflections on accelerating change | |||
| Why the AI Race Ends in Disaster (with Daniel Kokotajlo) | 03 juil. 2025 | 01:10:27 | |
On this episode, Daniel Kokotajlo joins me to discuss why artificial intelligence may surpass the transformative power of the Industrial Revolution, and just how much AI could accelerate AI research. We explore the implications of automated coding, the critical need for transparency in AI development, the prospect of AI-to-AI communication, and whether AI is an inherently risky technology. We end by discussing iterative forecasting and its role in anticipating AI's future trajectory. You can learn more about Daniel's work at: https://ai-2027.com and https://ai-futures.org Timestamps: 00:00:00 Preview and intro 00:00:50 Why AI will eclipse the Industrial Revolution 00:09:48 How much can AI speed up AI research? 00:16:13 Automated coding and diffusion 00:27:37 Transparency in AI development 00:34:52 Deploying AI internally 00:40:24 Communication between AIs 00:49:23 Is AI inherently risky? 00:59:54 Iterative forecasting | |||
| Preparing for an AI Economy (with Daniel Susskind) | 27 juin 2025 | 01:03:38 | |
On this episode, Daniel Susskind joins me to discuss disagreements between AI researchers and economists, how we can best measure AI’s economic impact, how human values can influence economic outcomes, what meaningful work will remain for humans in the future, the role of commercial incentives in AI development, and the future of education. You can learn more about Daniel's work here: https://www.danielsusskind.com Timestamps: 00:00:00 Preview and intro 00:03:19 AI researchers versus economists 00:10:39 Measuring AI's economic effects 00:16:19 Can AI be steered in positive directions? 00:22:10 Human values and economic outcomes 00:28:21 What will remain for people to do? 00:44:58 Commercial incentives in AI 00:50:38 Will education move towards general skills? 00:58:46 Lessons for parents | |||
| Will AI Companies Respect Creators' Rights? (with Ed Newton-Rex) | 20 juin 2025 | 01:27:15 | |
Ed Newton-Rex joins me to discuss the issue of AI models trained on copyrighted data, and how we might develop fairer approaches that respect human creators. We talk about AI-generated music, Ed’s decision to resign from Stability AI, the industry’s attitude towards rights, authenticity in AI-generated art, and what the future holds for creators, society, and living standards in an increasingly AI-driven world. Learn more about Ed's work here: https://ed.newtonrex.com Timestamps: 00:00:00 Preview and intro 00:04:18 AI-generated music 00:12:15 Resigning from Stability AI 00:16:20 AI industry attitudes towards rights 00:26:22 Fairly Trained 00:37:16 Special kinds of training data 00:50:42 The longer-term future of AI 00:56:09 Will AI improve living standards? 01:03:10 AI versions of artists 01:13:28 Authenticity and art 01:18:45 Competitive pressures in AI 01:24:06 Priorities going forward | |||
| AI Timelines and Human Psychology (with Sarah Hastings-Woodhouse) | 13 juin 2025 | 01:15:50 | |
On this episode, Sarah Hastings-Woodhouse joins me to discuss what benchmarks actually measure, AI’s development trajectory in comparison to other technologies, tasks that AI systems can and cannot handle, capability profiles of present and future AIs, the notion of alignment by default, and the leading AI companies’ vague AGI plans. We also discuss the human psychology of AI, including the feelings of living in the "fast world" versus the "slow world", and navigating long-term projects given short timelines. Timestamps: 00:00:00 Preview and intro 00:00:46 What do benchmarks measure? 00:08:08 Will AI develop like other tech? 00:14:13 Which tasks can AIs do? 00:23:00 Capability profiles of AIs 00:34:04 Timelines and social effects 00:42:01 Alignment by default? 00:50:36 Can vague AGI plans be useful? 00:54:36 The fast world and the slow world 01:08:02 Long-term projects and short timelines | |||
| Could Powerful AI Break Our Fragile World? (with Michael Nielsen) | 06 juin 2025 | 01:01:29 | |
On this episode, Michael Nielsen joins me to discuss how humanity's growing understanding of nature poses dual-use challenges, whether existing institutions and governance frameworks can adapt to handle advanced AI safely, and how we might recognize signs of dangerous AI. We explore the distinction between AI as agents and tools, how power is latent in the world, implications of widespread powerful hardware, and finally touch upon the philosophical perspectives of deep atheism and optimistic cosmism. Timestamps: 00:00:00 Preview and intro 00:01:05 Understanding is dual-use 00:05:17 Can we handle AI like other tech? 00:12:08 Can institutions adapt to AI? 00:16:50 Recognizing signs of dangerous AI 00:22:45 Agents versus tools 00:25:43 Power is latent in the world 00:35:45 Widespread powerful hardware 00:42:09 Governance mechanisms for AI 00:53:55 Deep atheism and optimistic cosmism | |||
| Facing Superintelligence (with Ben Goertzel) | 23 mai 2025 | 01:32:34 | |
On this episode, Ben Goertzel joins me to discuss what distinguishes the current AI boom from previous ones, important but overlooked AI research, simplicity versus complexity in the first AGI, the feasibility of alignment, benchmarks and economic impact, potential bottlenecks to superintelligence, and what humanity should do moving forward. Timestamps: 00:00:00 Preview and intro 00:01:59 Thinking about AGI in the 1970s 00:07:28 What's different about this AI boom? 00:16:10 Former taboos about AGI 00:19:53 AI research worth revisiting 00:35:53 Will the first AGI be simple? 00:48:49 Is alignment achievable? 01:02:40 Benchmarks and economic impact 01:15:23 Bottlenecks to superintelligence 01:23:09 What should we do? | |||
| Will Future AIs Be Conscious? (with Jeff Sebo) | 16 mai 2025 | 01:34:28 | |
On this episode, Jeff Sebo joins me to discuss artificial consciousness, substrate-independence, possible tensions between AI risk and AI consciousness, the relationship between consciousness and cognitive complexity, and how intuitive versus intellectual approaches guide our understanding of these topics. We also discuss AI companions, AI rights, and how we might measure consciousness effectively. You can follow Jeff’s work here: https://jeffsebo.net/ Timestamps: 00:00:00 Preview and intro 00:02:56 Imagining artificial consciousness 00:07:51 Substrate-independence? 00:11:26 Are we making progress? 00:18:03 Intuitions about explanations 00:24:43 AI risk and AI consciousness 00:40:01 Consciousness and cognitive complexity 00:51:20 Intuition versus intellect 00:58:48 AIs as companions 01:05:24 AI rights 01:13:00 Acting under time pressure 01:20:16 Measuring consciousness 01:32:11 How can you help? | |||
| Understanding AI Agents: Time Horizons, Sycophancy, and Future Risks (with Zvi Mowshowitz) | 09 mai 2025 | 01:35:10 | |
On this episode, Zvi Mowshowitz joins me to discuss sycophantic AIs, bottlenecks limiting autonomous AI agents, and the true utility of benchmarks in measuring progress. We then turn to time horizons of AI agents, the impact of automating scientific research, and constraints on scaling inference compute. Zvi also addresses humanity’s uncertain AI-driven future, the unique features setting AI apart from other technologies, and AI’s growing influence in financial trading. You can follow Zvi's excellent blog here: https://thezvi.substack.com Timestamps: 00:00:00 Preview and introduction 00:02:01 Sycophantic AIs 00:07:28 Bottlenecks for AI agents 00:21:26 Are benchmarks useful? 00:32:39 AI agent time horizons 00:44:18 Impact of automating research 00:53:00 Limits to scaling inference compute 01:02:51 Will the future go well for humanity? 01:12:22 A good plan for safe AI 01:26:03 What makes AI different? 01:31:29 AI in trading | |||
| Inside China's AI Strategy: Innovation, Diffusion, and US Relations (with Jeffrey Ding) | 25 avr. 2025 | 01:02:33 | |
On this episode, Jeffrey Ding joins me to discuss diffusion of AI versus AI innovation, how US-China dynamics shape AI’s global trajectory, and whether there is an AI arms race between the two powers. We explore Chinese attitudes toward AI safety, the level of concentration of AI development, and lessons from historical technology diffusion. Jeffrey also shares insights from translating Chinese AI writings and the potential of automating translations to bridge knowledge gaps. You can learn more about Jeffrey’s work at: https://jeffreyjding.github.io Timestamps: 00:00:00 Preview and introduction 00:01:36 A US-China AI arms race? 00:10:58 Attitudes to AI safety in China 00:17:53 Diffusion of AI 00:25:13 Innovation without diffusion 00:34:29 AI development concentration 00:41:40 Learning from the history of technology 00:47:48 Translating Chinese AI writings 00:55:36 Automating translation of AI writings | |||
| How Will We Cooperate with AIs? (with Allison Duettmann) | 11 avr. 2025 | 01:36:03 | |
On this episode, Allison Duettmann joins me to discuss centralized versus decentralized AI, how international governance could shape AI’s trajectory, how we might cooperate with future AIs, and the role of AI in improving human decision-making. We also explore which lessons from history apply to AI, the future of space law and property rights, whether technology is invented or discovered, and how AI will impact children. You can learn more about Allison's work at: https://foresight.org Timestamps: 00:00:00 Preview 00:01:07 Centralized AI versus decentralized AI 00:13:02 Risks from decentralized AI 00:25:39 International AI governance 00:39:52 Cooperation with future AIs 00:53:51 AI for decision-making 01:05:58 Capital intensity of AI 01:09:11 Lessons from history 01:15:50 Future space law and property rights 01:27:28 Is technology invented or discovered? 01:32:34 Children in the age of AI | |||
| Brain-like AGI and why it's Dangerous (with Steven Byrnes) | 04 avr. 2025 | 01:13:14 | |
On this episode, Steven Byrnes joins me to discuss brain-like AGI safety. We discuss learning versus steering systems in the brain, the distinction between controlled AGI and social-instinct AGI, why brain-inspired approaches might be our most plausible route to AGI, and honesty in AI models. We also talk about how people can contribute to brain-like AGI safety and compare various AI safety strategies. You can learn more about Steven's work at: https://sjbyrnes.com/agi.html Timestamps: 00:00 Preview 00:54 Brain-like AGI Safety 13:16 Controlled AGI versus Social-instinct AGI 19:12 Learning from the brain 28:36 Why is brain-like AI the most likely path to AGI? 39:23 Honesty in AI models 44:02 How to help with brain-like AGI safety 53:36 AI traits with both positive and negative effects 01:02:44 Different AI safety strategies | |||