Future of Life Institute Podcast – Details, episodes & analysis

Nicolas Berggruen, investor and philanthropist, joins us to explore the dynamics of power, wisdom, technology and ideas in the 21st century. Topics discussed in this episode include: -What wisdom consists of -The role of ideas in society and civilization  -The increasing concentration of power and wealth -The technological displacement of human labor -Democracy, universal basic income, and universal basic capital  -Living an examined life You can find the page for this podcast here: https://futureoflife.org/2021/05/31/nicolas-berggruen-on-the-dynamics-of-power-wisdom-technology-and-ideas-in-the-age-of-ai/ Check out Nicolas' thoughts archive here: www.nicolasberggruen.com Have any feedback about the podcast? You can share your thoughts here: www.surveymonkey.com/r/DRBFZCT Timestamps:  0:00 Intro 1:45 The race between the power of our technology and the wisdom with which we manage it 5:19 What is wisdom?  8:30 The power of ideas  11:06 Humanity’s investment in wisdom vs the power of our technology  15:39 Why does our wisdom lag behind our power?  20:51 Technology evolving into an agent  24:28 How ideas play a role in the value alignment of technology  30:14 Wisdom for building beneficial AI and mitigating the race to power  34:37 Does Mark Zuckerberg have control of Facebook?  36:39 Safeguarding the human mind and maintaining control of AI  42:26 The importance of the examined life in the 21st century  45:56 An example of the examined life  48:54 Important ideas for the 21st century  52:46 The concentration of power and wealth, and a proposal for universal basic capital  1:03:07 Negative and positive futures  1:06:30 Final thoughts from Nicolas This podcast is possible because of the support of listeners like you. If you found this conversation to be meaningful or valuable, consider supporting it directly by donating at futureoflife.org/donate. Contributions like yours make these conversations possible.

Bart Selman, Professor of Computer Science at Cornell University, joins us to discuss a wide range of AI issues, from autonomous weapons and AI consciousness to international governance and the possibilities of superintelligence. Topics discussed in this episode include: -Negative and positive outcomes from AI in the short, medium, and long-terms -The perils and promises of AGI and superintelligence -AI alignment and AI existential risk -Lethal autonomous weapons -AI governance and racing to powerful AI systems -AI consciousness You can find the page for this podcast here: https://futureoflife.org/2021/05/20/bart-selman-on-the-promises-and-perils-of-artificial-intelligence/ Have any feedback about the podcast? You can share your thoughts here: www.surveymonkey.com/r/DRBFZCT Timestamps:  0:00 Intro  1:35 Futures that Bart is excited about                   4:08 Positive futures in the short, medium, and long-terms 7:23 AGI timelines  8:11 Bart’s research on “planning” through the game of Sokoban 13:10 If we don’t go extinct, is the creation of AGI and superintelligence inevitable?  15:28 What’s exciting about futures with AGI and superintelligence?  17:10 How long does it take for superintelligence to arise after AGI?  21:08 Would a superintelligence have something intelligent to say about income inequality?  23:24 Are there true or false answers to moral questions?  25:30 Can AGI and superintelligence assist with moral and philosophical issues? 28:07 Do you think superintelligences converge on ethics?  29:32 Are you most excited about the short or long-term benefits of AI?  34:30 Is existential risk from AI a legitimate threat?  35:22 Is the AI alignment problem legitimate?  43:29 What are futures that you fear?  46:24 Do social media algorithms represent an instance of the alignment problem?  51:46 The importance of educating the public on AI  55:00 Income inequality, cyber security, and negative futures  1:00:06 Lethal autonomous weapons  1:01:50 Negative futures in the long-term  1:03:26 How have your views of AI alignment evolved?  1:06:53 Bart’s plans and intentions for the Association for the Advancement of Artificial Intelligence 1:13:45 Policy recommendations for existing AIs and the AI ecosystem  1:15:35 Solving the parts of the AI alignment that won’t be solved by industry incentives  1:18:17 Narratives of an international race to powerful AI systems  1:20:42 How does an international race to AI affect the chances of successful AI alignment?  1:23:20 Is AI a zero sum game?  1:28:51 Lethal autonomous weapons governance  1:31:38 Does the governance of autonomous weapons affect outcomes from AGI  1:33:00 AI consciousness  1:39:37 Alignment is important and the benefits of AI can be great This podcast is possible because of the support of listeners like you. If you found this conversation to be meaningful or valuable, consider supporting it directly by donating at futureoflife.org/donate. Contributions like yours make these conversations possible.

Jaan Tallinn, investor, programmer, and co-founder of the Future of Life Institute, joins us to discuss his perspective on AI, synthetic biology, unknown unknows, and what's needed for mitigating existential risk in the 21st century. Topics discussed in this episode include: -Intelligence and coordination -Existential risk from AI, synthetic biology, and unknown unknowns -AI adoption as a delegation process -Jaan's investments and philanthropic efforts -International coordination and incentive structures -The short-term and long-term AI safety communities You can find the page for this podcast here: https://futureoflife.org/2021/04/20/jaan-tallinn-on-avoiding-civilizational-pitfalls-and-surviving-the-21st-century/ Have any feedback about the podcast? You can share your thoughts here: www.surveymonkey.com/r/DRBFZCT Timestamps:  0:00 Intro 1:29 How can humanity improve? 3:10 The importance of intelligence and coordination 8:30 The bottlenecks of input and output bandwidth as well as processing speed between AIs and humans 15:20 Making the creation of AI feel dangerous and how the nuclear power industry killed itself by downplaying risks 17:15 How Jaan evaluates and thinks about existential risk 18:30 Nuclear weapons as the first existential risk we faced 20:47 The likelihood of unknown unknown existential risks 25:04 Why Jaan doesn't see nuclear war as an existential risk 27:54 Climate change 29:00 Existential risk from synthetic biology 31:29 Learning from mistakes, lacking foresight, and the importance of generational knowledge 36:23 AI adoption as a delegation process 42:52 Attractors in the design space of AI 44:24 The regulation of AI 45:31 Jaan's investments and philanthropy in AI 55:18 International coordination issues from AI adoption as a delegation process 57:29 AI today and the negative impacts of recommender algorithms 1:02:43 Collective, institutional, and interpersonal coordination 1:05:23 The benefits and risks of longevity research 1:08:29 The long-term and short-term AI safety communities and their relationship with one another 1:12:35 Jaan's current philanthropic efforts 1:16:28 Software as a philanthropic target 1:19:03 How do we move towards beneficial futures with AI? 1:22:30 An idea Jaan finds meaningful 1:23:33 Final thoughts from Jaan 1:25:27 Where to find Jaan This podcast is possible because of the support of listeners like you. If you found this conversation to be meaningful or valuable, consider supporting it directly by donating at futureoflife.org/donate. Contributions like yours make these conversations possible.

Joscha Bach, Cognitive Scientist and AI researcher, as well as Anthony Aguirre, UCSC Professor of Physics, join us to explore the world through the lens of computation and the difficulties we face on the way to beneficial futures.  Topics discussed in this episode include: -Understanding the universe through digital physics -How human consciousness operates and is structured -The path to aligned AGI and bottlenecks to beneficial futures -Incentive structures and collective coordination You can find the page for this podcast here: https://futureoflife.org/2021/03/31/joscha-bach-and-anthony-aguirre-on-digital-physics-and-moving-towards-beneficial-futures/ You can find FLI's three new policy focused job postings here: futureoflife.org/job-postings/ Have any feedback about the podcast? You can share your thoughts here: www.surveymonkey.com/r/DRBFZCT Timestamps:  0:00 Intro 3:17 What is truth and knowledge? 11:39 What is subjectivity and objectivity? 14:32 What is the universe ultimately? 19:22 Is the universe a cellular automaton? Is the universe ultimately digital or analogue? 24:05 Hilbert's hotel from the point of view of computation 35:18 Seeing the world as a fractal 38:48 Describing human consciousness 51:10 Meaning, purpose, and harvesting negentropy 55:08 The path to aligned AGI 57:37 Bottlenecks to beneficial futures and existential security 1:06:53 A future with one, several, or many AGI systems? How do we maintain appropriate incentive structures? 1:19:39 Non-duality and collective coordination 1:22:53 What difficulties are there for an idealist worldview that involves computation? 1:27:20 Which features of mind and consciousness are necessarily coupled and which aren't? 1:36:40 Joscha's final thoughts on AGI This podcast is possible because of the support of listeners like you. If you found this conversation to be meaningful or valuable, consider supporting it directly by donating at futureoflife.org/donate. Contributions like yours make these conversations possible.

Roman Yampolskiy, Professor of Computer Science at the University of Louisville, joins us to discuss whether we can control, comprehend, and explain AI systems, and how this constrains the project of AI safety.  Topics discussed in this episode include: -Roman’s results on the unexplainability, incomprehensibility, and uncontrollability of AI -The relationship between AI safety, control, and alignment -Virtual worlds as a proposal for solving multi-multi alignment -AI security You can find the page for this podcast here: https://futureoflife.org/2021/03/19/roman-yampolskiy-on-the-uncontrollability-incomprehensibility-and-unexplainability-of-ai/ You can find FLI's three new policy focused job postings here: https://futureoflife.org/job-postings/ Have any feedback about the podcast? You can share your thoughts here: www.surveymonkey.com/r/DRBFZCT Timestamps:  0:00 Intro  2:35 Roman’s primary research interests  4:09 How theoretical proofs help AI safety research  6:23 How impossibility results constrain computer science systems 10:18 The inability to tell if arbitrary code is friendly or unfriendly  12:06 Impossibility results clarify what we can do  14:19 Roman’s results on unexplainability and incomprehensibility  22:34 Focusing on comprehensibility  26:17 Roman’s results on uncontrollability  28:33 Alignment as a subset of safety and control  30:48 The relationship between unexplainability, incomprehensibility, and uncontrollability with each other and with AI alignment  33:40 What does it mean to solve AI safety?  34:19 What do the impossibility results really mean?  37:07 Virtual worlds and AI alignment  49:55 AI security and malevolent agents  53:00 Air gapping, boxing, and other security methods  58:43 Some examples of historical failures of AI systems and what we can learn from them  1:01:20 Clarifying impossibility results 1:06 55 Examples of systems failing and what these demonstrate about AI  1:08:20 Are oracles a valid approach to AI safety?  1:10:30 Roman’s final thoughts This podcast is possible because of the support of listeners like you. If you found this conversation to be meaningful or valuable, consider supporting it directly by donating at futureoflife.org/donate. Contributions like yours make these conversations possible.

Stuart Russell, Professor of Computer Science at UC Berkeley, and Zachary Kallenborn, WMD and drone swarms expert, join us to discuss the highest risk and most destabilizing aspects of lethal autonomous weapons.  Topics discussed in this episode include: -The current state of the deployment and development of lethal autonomous weapons and swarm technologies -Drone swarms as a potential weapon of mass destruction -The risks of escalation, unpredictability, and proliferation with regards to autonomous weapons -The difficulty of attribution, verification, and accountability with autonomous weapons -Autonomous weapons governance as norm setting for global AI issues You can find the page for this podcast here: https://futureoflife.org/2021/02/25/stuart-russell-and-zachary-kallenborn-on-drone-swarms-and-the-riskiest-aspects-of-lethal-autonomous-weapons/ You can check out the new lethal autonomous weapons website here: https://autonomousweapons.org/ Have any feedback about the podcast? You can share your thoughts here: www.surveymonkey.com/r/DRBFZCT Timestamps:  0:00 Intro 2:23 Emilia Javorsky on lethal autonomous weapons 7:27 What is a lethal autonomous weapon? 11:33 Autonomous weapons that exist today 16:57 The concerns of collateral damage, accidental escalation, scalability, control, and error risk 26:57 The proliferation risk of autonomous weapons 32:30 To what extent are global superpowers pursuing these weapons? What is the state of industry's pursuit of the research and manufacturing of this technology 42:13 A possible proposal for a selective ban on small anti-personnel autonomous weapons 47:20 Lethal autonomous weapons as a potential weapon of mass destruction 53:49 The unpredictability of autonomous weapons, especially when swarms are interacting with other swarms 58:09 The risk of autonomous weapons escalating conflicts 01:10:50 The risk of drone swarms proliferating 01:20:16 The risk of assassination 01:23:25 The difficulty of attribution and accountability 01:26:05 The governance of autonomous weapons being relevant to the global governance of AI 01:30:11 The importance of verification for responsibility, accountability, and regulation 01:35:50 Concerns about the beginning of an arms race and the need for regulation 01:38:46 Wrapping up 01:39:23 Outro This podcast is possible because of the support of listeners like you. If you found this conversation to be meaningful or valuable, consider supporting it directly by donating at futureoflife.org/donate. Contributions like yours make these conversations possible.

John Prendergast, former adjunct professor of psychology at the California Institute of Integral Studies, joins Lucas Perry for a discussion about the experience and effects of ego-identification, how to shift to new levels of identity, the nature of non-dual awareness, and the potential relationship between waking up and collective human problems. This is not an FLI Podcast, but a special release where Lucas shares a direction he feels has an important relationship with AI alignment and existential risk issues. Topics discussed in this episode include: -The experience of egocentricity and ego-identification -Waking up into heart awareness -The movement towards and qualities of non-dual consciousness -The ways in which the condition of our minds collectively affect the world -How waking up may be relevant to the creation of AGI You can find the page for this podcast here: https://futureoflife.org/2021/02/09/john-prendergast-on-non-dual-awareness-and-wisdom-for-the-21st-century/ Have any feedback about the podcast? You can share your thoughts here: https://www.surveymonkey.com/r/DRBFZCT Timestamps:  0:00 Intro 7:10 The modern human condition 9:29 What egocentricity and ego-identification are 15:38 Moving beyond the experience of self 17:38 The origins and structure of self 20:25 A pointing out instruction for noticing ego-identification and waking up out of it 24:34 A pointing out instruction for abiding in heart-mind or heart awareness 28:53 The qualities of and moving into heart awareness and pure awareness 33:48 An explanation of non-dual awareness 40:50 Exploring the relationship between awareness, belief, and action 46:25 Growing up and improving the egoic structure 48:29 Waking up as recognizing true nature 51:04 Exploring awareness as primitive and primary 53:56 John's dream of Sri Nisargadatta Maharaj 57:57 The use and value of conceptual thought and the mind 1:00:57 The epistemics of heart-mind and the conceptual mind as we shift levels of identity 1:17:46 A pointing out instruction for inquiring into core beliefs 1:27:28 The universal heart, qualities of awakening, and the ethical implications of such shifts 1:31:38 Wisdom, waking up, and growing up for the transgenerational issues of the 21st century 1:38:44 Waking up and its applicability to the creation of AGI 1:43:25 Where to find, follow, and reach out to John 1:45:56 Outro This podcast is possible because of the support of listeners like you. If you found this conversation to be meaningful or valuable, consider supporting it directly by donating at futureoflife.org/donate. Contributions like yours make these conversations possible.