A biweekly podcast where hosts Nathan Labenz and Erik Torenberg interview the builders on the edge of AI and explore the dramatic shift it will unlock in the coming years.
The Cognitive Revolution is part of the Turpentine podcast network. To learn more: turpentine.co
Site
RSS
Apple
Recent rankings
Latest chart positions across Apple Podcasts and Spotify rankings.
Shared links between episodes and podcasts
Links found in episode descriptions and other podcasts that share them.
Computational Life: How Self-Replicators Arise from Randomness, with Google’s Ettore Randazzo and Luca Versari
Season 1
Friday, August 30, 2024 • Duration 01:30:00
Nathan explores the emergence of computational life with Google researchers Ettore Randazzo and Luca Versari. In this episode of The Cognitive Revolution, we delve into their groundbreaking paper on self-replicating programs arising from simple interactions. Join us for a fascinating discussion on the implications for AI development, the origins of life, and the potential future of artificial intelligence.
Apply to join over 400 founders and execs in the Turpentine Network: https://hmplogxqz0y.typeform.com/to/JCkphVqj
RECOMMENDED PODCAST:
🎙️ This Won't Last - Eavesdrop on Keith Rabois, Kevin Ryan, Logan Bartlett, and Zach Weinberg's monthly backchannel. They unpack their hottest takes on the future of tech, business, venture, investing, and politics.
Apple Podcasts: https://podcasts.apple.com/us/podcast/id1765665937
Spotify: https://open.spotify.com/show/2HwSNeVLL1MXy0RjFPyOSz
YouTube: https://www.youtube.com/@ThisWontLastpodcast
SPONSORS:
Building an enterprise-ready SaaS app? WorkOS has got you covered with easy-to-integrate APIs for SAML, SCIM, and more. Join top startups like Vercel, Perplexity, Jasper & Webflow in powering your app with WorkOS. Enjoy a free tier for up to 1M users! Start now at https://bit.ly/WorkOS-Turpentine-Network
80,000 Hours offers free one-on-one career advising for Cognitive Revolution listeners aiming to tackle global challenges, especially in AI. They connect high-potential individuals with experts, opportunities, and personalized career plans to maximize positive impact. Apply for a free call at https://80000hours.org/cognitiverevolution to accelerate your career and contribute to solving pressing AI-related issues.
The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR
Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/
Head to Squad to access global engineering without the headache and at a fraction of the cost: head to https://choosesquad.com/ and mention “Turpentine” to skip the waitlist.
CHAPTERS:
(00:00:00) About the Show
(00:00:22) Sponsor: WorkOS
(00:01:22) About the Episode
(00:04:02) Introduction and Paper Overview
(00:05:38) Self-Replication and Life
(00:07:59) Complexity and Information Theory
(00:17:09) Sponsors: 80,000 Hours | Brave
(00:19:44) Experiment Setup
(00:24:27) Evolution of Self-Replicators
(00:34:41) Sponsors: Omneky | Squad
(00:36:07) Types of Self-Replicators
(00:38:23) Symbiosis and Parasitic Behaviors
(00:46:17) Implications for Life in the Universe
(01:05:47) Intelligence and Complexity in AI
(01:21:03) Concluding Thoughts
Scaling Forecasting: AI Forecasting Tournaments & the Road to Epistemic Security, with Deger Turan, CEO of Metaculus
Season 1
Wednesday, August 28, 2024 • Duration 01:53:21
Nathan explores the cutting-edge world of AI-powered forecasting with Deger Turan, CEO of Metaculus. In this episode of The Cognitive Revolution, we discuss how AI is revolutionizing prediction markets, the potential for AI to outperform human forecasters, and Metaculus's ambitious new AI forecasting benchmark tournament. Join us for an insightful conversation about the future of decision-making and collective intelligence.
Participate in Metaculus' first of its kind bot forecasting tournament: https://www.metaculus.com/aib/
Apply to join over 400 founders and execs in the Turpentine Network: https://hmplogxqz0y.typeform.com/to/JCkphVqj
RECOMMENDED PODCAST:
🎙️ This Won't Last - Eavesdrop on Keith Rabois, Kevin Ryan, Logan Bartlett, and Zach Weinberg's monthly backchannel. They unpack their hottest takes on the future of tech, business, venture, investing, and politics.
Apple Podcasts: https://podcasts.apple.com/us/podcast/id1765665937
Spotify: https://open.spotify.com/show/2HwSNeVLL1MXy0RjFPyOSz
YouTube: https://www.youtube.com/@ThisWontLastpodcast
SPONSORS:
Building an enterprise-ready SaaS app? WorkOS has got you covered with easy-to-integrate APIs for SAML, SCIM, and more. Join top startups like Vercel, Perplexity, Jasper & Webflow in powering your app with WorkOS. Enjoy a free tier for up to 1M users! Start now at https://bit.ly/WorkOS-Turpentine-Network
80,000 Hours offers free one-on-one career advising for Cognitive Revolution listeners aiming to tackle global challenges, especially in AI. They connect high-potential individuals with experts, opportunities, and personalized career plans to maximize positive impact. Apply for a free call at https://80000hours.org/cognitiverevolution to accelerate your career and contribute to solving pressing AI-related issues.
The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR
Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/
Head to Squad to access global engineering without the headache and at a fraction of the cost: head to https://choosesquad.com/ and mention “Turpentine” to skip the waitlist.
CHAPTERS:
(00:00:00) About the Show
(00:00:22) Sponsor: WorkOS
(00:01:22) About the Episode
(00:04:47) Introduction and Background
(00:08:42) Deger's Journey to Forecasting
(00:13:06) Current State of Forecasting
(00:20:23) Sponsors: 80,000 Hours | Brave
(00:22:55) Overview of Metaculous
(00:23:30) AI Forecasting Research Papers
(00:35:44) Sponsors: Omneky | Squad
(00:37:31) AI Forecasting Benchmark Tournament
(00:44:01) Improving Forecasting Accuracy
(00:51:56) Scoring System and Tournament Rules
(01:04:17) AI Forecasting Benchmark Series Details
(01:11:03) Tournament Structure and Participation
(01:17:07) Consistency in AI Forecasting
(01:34:36) Risks and Challenges in Forecasting
(01:42:43) Consensus Illusion and Policy Making
(01:53:55) Outro
Zooming Out on AI, from the Nick Halaris Show
Season 1
Saturday, July 27, 2024 • Duration 01:32:18
Nathan Labenz joins the Nick Halaris Show to discuss the current AI landscape, balancing enthusiasm with critical analysis. This episode of The Cognitive Revolution explores AI's transformative potential in medicine, law, and automation, while addressing concerns about safety, control, and geopolitical implications. Join us for an insightful conversation on responsible AI development and its impact on society.
Apply to join over 400 founders and execs in the Turpentine Network: https://hmplogxqz0y.typeform.com/to/JCkphVqj
RECOMMENDED PODCAST:
🎙️ Second Opinion - A new podcast for health-tech insiders from Christina Farr of the Second Opinion newsletter. Join Christina Farr, Luba Greenwood, and Ash Zenooz every week as they challenge industry experts with tough questions about the best bets in health-tech.
Apple Podcasts: https://podcasts.apple.com/us/podcast/id1759267211
Spotify: https://open.spotify.com/show/0A8NwQE976s32zdBbZw6bv
-
🎙️ History 102 with WhatifAltHist
Every week, creator of WhatifAltHist Rudyard Lynch and Erik Torenberg cover a major topic in history in depth -- in under an hour. This season will cover classical Greece, early America, the Vikings, medieval Islam, ancient China, the fall of the Roman Empire, and more.
Subscribe on Spotify: https://open.spotify.com/show/36Kqo3BMMUBGTDo1IEYihm
Apple: https://podcasts.apple.com/us/podcast/history-102-with-whatifalthists-rudyard-lynch-and/id1730633913
YouTube: https://www.youtube.com/@History102-qg5oj
SPONSORS:
Building an enterprise-ready SaaS app? WorkOS has got you covered with easy-to-integrate APIs for SAML, SCIM, and more. Join top startups like Vercel, Perplexity, Jasper & Webflow in powering your app with WorkOS. Enjoy a free tier for up to 1M users! Start now at https://bit.ly/WorkOS-TCR
Oracle Cloud Infrastructure (OCI) is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds; offers one consistent price, and nobody does data better than Oracle. If you want to do more and spend less, take a free test drive of OCI at https://oracle.com/cognitive
The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR
Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/
Head to Squad to access global engineering without the headache and at a fraction of the cost: head to https://choosesquad.com/ and mention “Turpentine” to skip the waitlist.
CHAPTERS:
(00:00:00) About the Show
(00:00:25) Sponsor: WorkOS
(00:01:25) About the Show
(00:03:36) Intro
(00:05:53) How did you get into AI?
(00:11:09) The power of the transformer
(00:14:16) Waymark growth
(00:19:00) Sponsors: Oracle | Brave
(00:21:04) The potential benefits of AI
(00:34:20) AI and mass inequality
(00:35:26) AI could lead to Universal Basic Income
(00:37:37) AI supply and the future of work
(00:40:53) The AI future
(00:43:38) The new social contract
(00:47:06) Optimism about the future
(00:53:19) What are people afraid of?
(00:58:31) Reinforcement learning from human feedback
(01:07:56) What did Ilya see?
(01:10:23) Who controls AI?
(01:13:23) China
(01:16:05) Taiwan and the chip sector
(01:20:05) The AI arms race
(01:23:22) AI Governance
(01:26:28) AI Safety
(01:29:22) Custom Models
(01:31:33) Closing
(01:32:23) Outro
AI Engineers, Pendants, and Competition Between OpenAI and Developers with Swyx of Latent Space
Season 1 · Episode 68
Thursday, October 5, 2023 • Duration 01:33:20
In this episode, Nathan sits down with Swyx of Latent Space to chat about AI engineers and tools to check out, competitive dynamics between OpenAI, other foundation model providers, and developers, and if Swyx would wear an AI Horcrux. We also learn that Nathan lived in the same dorm as Mark Zuckerberg back in his college days (and was a late adopter to Facebook…). If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive
Subscribe to the show Substack to join future conversations and submit questions to upcoming guests! https://cognitiverevolution.substack.com/
We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.
SPONSORS: Shopify | NetSuite | Omneky
Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive
NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.
Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.
TIMESTAMPS:
(00:00) Episode Preview
(00:00:49) AI Nathan’s intro
(00:03:14) What is an AI engineer?
(00:05:56) What backgrounds do AI engineers typically have?
(00:15:51) Sponsors: Netsuite | Omneky
(00:17:13) Swyx’s Discord AI project
(00:20:41) Key tools for AI engineers
(00:23:42) HumanLoop, Guardrails, Langchain
(00:27:01) Criteria for identifying capable AI engineers when hiring
(00:30:59) Skepticism around AI being a fad and doubts about contributing to AI
(00:34:03) AI Engineer Conference speaker lineup
(00:41:14) AI agents and two years to AGI
(00:46:04) Expectations and disagreement around what AI agent capabilities will work soon
(00:50:12) Swyx’s OpenAI thesis
(00:53:03) AI safety considerations and the role of AI engineers
(00:56:24) Disagreement on whether AI will soon be able to generate code pull requests
(01:01:07) AI helping non-technical people to code
(01:01:49) Multi-modal Chat-GPT and the future implications
(01:03:33) Nathan living in the same dorm as Mark Zuckerberg
(01:04:44) Competitive dynamics between OpenAI and other AI model developers
(01:05:39) Play.ht vs ElevenLabs
(01:09:20) The tension between platforms and developers building on top of them
(01:11:40) The best thing startups can do to compete with foundation model providers
(01:16:26) User identity/authentication services like Login with OpenAI
(01:19:20) Google vs the other live players
(01:20:46) AI Horcruxes / Pendants
(01:22:05) The concept of an AI app bundle for consumers and developers
Gunning for Google with Perplexity CEO Aravind Srinivas
Tuesday, October 3, 2023 • Duration 59:33
In this episode, Aravind Srinivas of Perplexity AI returns to the show. They discuss Perplexity’s growth to millions of queries per day, CEO Aravind’s favourite Perplexity use cases, and how Perplexity ships fast in a competitive landscape against Google and major AI live players creating their own search engines. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive
We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.
SPONSORS: Shopify | NetSuite | Omneky
Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive
NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.
Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.
X:
@labenz (Nathan)
@AravSrinivas (Aravind)
@eriktorenberg (Erik)
@cogrev_podcast
LINKS:
- Perplexity AI: https://www.perplexity.ai/
- Part 1 with Aravind: https://www.youtube.com/watch?v=YSYQZZu4MEM
TIMESTAMPS:
(00:00:00) - Episode Preview
(00:01:12) - Introducing Aravind Srinivas and Perplexity AI
(00:05:05) - Perplexity's growth to millions of queries per day
(00:06:21) - Perplexity's retention and market share vs. ChatGPT
(00:08:32) - Improving reliability, speed, accuracy, and new features
(00:10:33) - Getting to daily usage from weekly usage
(00:13:21) - Aravind's favorite Perplexity use cases
(00:18:34) - Issues with lack of sufficient web coverage for hard queries
(00:18:38) - Sponsors: Netsuite | Omneky
(00:23:13) - Building their own search index and models for self-sufficiency
(00:26:29) - Using their own index vs real-time web queries
(00:28:43) - Integrating default search and Copilot companion experiences
(00:31:49) - Training their own models as the only viable long-term option
(00:36:56) - The web becoming more like an API accessed via AI assistants
(00:41:23) - Not trying to copy Google's business model
(00:44:36) - Spending more on compute than people to get scale efficiencies
(00:49:35) - Replacing all parts of the stack with their own proprietary versions
(00:53:59) - Shipping fast out of necessity in a competitive war zone
(00:58:00) - AI assistants becoming a significant part of human digital activity
The Cognitive Revolution is brought to you by the Turpentine Media network.
Producer: Vivian Meng
Executive Producers: Amelia Salyers, and Erik Torenberg
Editor: Graham Bessellieu
For inquiries about guests or sponsoring the podcast, please email vivian@turpentine.co
OpenAI, Amazon's Anthropic Investment, and the Roman Empire with Zvi Mowshowitz
Season 1 · Episode 65
Friday, September 29, 2023 • Duration 01:53:54
Zvi Mowshowitz, the writer behind Don't Worry About the Vase, returns to catch up with Nathan on everything OpenAI, Amazon-Anthropic collab, and Google Deepmind. They also discuss Perplexity, deepfakes, and software bundling vs the Roman Empire. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive
Definitely also take a moment to subscribe to Zvi's blog Don't Worry About the Vase
(https://thezvi.wordpress.com/) - Zvi is an information hyperprocessor who synthesizes vast amounts of new and ever-evolving information into extremely clear summaries that help educated people keep up with the latest news.
We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.
SPONSORS: Shopify | NetSuite | Omneky
Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive
NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.
Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.
TIMESTAMPS:
00:00:00 - Episode Preview
(00:02:42) - Nathan's experience using Code Interpreter for a React app
00:06:09 - Zvi's perspective on Code Interpreter and other new Anthropic products
(00:10:47) - Nathan's approach of "coding by analogy" using Code Interpreter
(00:15:42) - Sponsors: Netsuite | Omneky
(00:17:00 )- Performance degradation issues with large context windows
(00:22:50) - The disconnect between Anthropic's value and what users are willing to pay
(00:31:56) - Predicting Gemini's capabilities relative to GPT-4
00:30:13 - Rating Code Interpreter's capabilities
(00:37:53) - Using Perplexity vs. Anthropic products for search
(00:44:11) - Potential for a bundled subscription for multiple AI services
(00:47:39) - Challenges of getting competitors to agree to bundling
(00:54:05) - Concerns over information pollution from synthetic content
(00:56:36) - Filtering adversarial vs. unintentional bogus information
(01:02:20) - Dangers of info pollution visible in Archive dataset
(01:11:15) - Kevin Fisher’s AI Souls demo with emotional voices
(01:12:15) - Difficulty of detecting AI voices/images for a general audience
(01:14:32) - Being optimistic about defending against deepfakes
(01:21:12) - The reversal curse in language models
(01:46:12) - Implications of Amazon investing in Anthropic
(01:49:20) - Non-standard terms likely affected the Anthropic valuation
(01:51:13) - Survey of the AI Safety landscape
OpenAI Fine-Tuning Update, Acceleration Debate, and Bundling AI Services
Season 1 · Episode 64
Tuesday, September 26, 2023 • Duration 01:17:49
Nathan and Erik chat OpenAI GPT-3.5 fine tuning updates, using GPT 4 outputs to fine-tune 3.5, when to accelerate, and AI bundles. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive
We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.
SPONSORS: Shopify | NetSuite | Omneky
Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive
NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.
Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.
TIMESTAMPS:
(00:00) - Intro and GPT-3.5 fine tuning update
(05:50) - Using GPT-4 to generate training data for GPT-3.5
(09:54) - Potential for training models on synthetic/cleaned data
(10:24) - Using chain of thought prompts to improve model performance
(15:02) - Sponsors: Netsuite | Omneky
(19:39) - Accelerating applications vs new AI models
(20:40) - What to accelerate vs what to slow down
(26:07) - AI as a co-pilot vs fully automating tasks
(30:12) - When to delegate to AI
(36:40)- Displacement of human roles by AI systems
(40:39) - Does training on synthetic data solve a problem?
(45:10) - The idea of an AI bundle/subscription
(50:28) - Bundling in other industries like cable and SaaS
(54:27) - Churn and retention challenges for AI apps
(01:02:54) - Low retention for easy-to-use AI apps
(01:03:57) - Incentives for AI companies to join a bundle
(01:04:39) - Potential for collaboration between AI companies
(01:12:01) - Leading AI firms creating separate bundles
(01:16:43) - Outro
Universal Jailbreaks with Zico Kolter, Andy Zou, and Asher Trockman
Season 1 · Episode 63
Friday, September 22, 2023 • Duration 02:16:07
In this episode, Nathan sits down with three researchers at Carnegie Mellon studying adversarial attacks and mimetic initialization: Zico Kolter, Andy Zou, and Asher Trockman. They discuss: the motivation behind researching universal adversarial attacks on language models, how the attacks work, and the short term harms and long term risks of these jailbreaks. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive
We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.
TIMESTAMPS:
[00:00:00] - Introducing the podcast and guests Zico Kolter, Andy Zou, and Asher Trockman
[00:06:32] - Discussing the motivation and high-level strategy for the universal adversarial attack on language models
[00:09:33] - Explaining how the attacks work by adding nonsense tokens to maximize target sequence probability
[00:11:06] - Comparing to prior adversarial attacks in vision models
[00:13:47] - Details on the attack optimization process and discrete token search
[00:17:09] - The empirical notion of "mode switching" in the language models
[00:21:18] - Technical details on gradient computation across multiple models and prompts
[00:23:46] - Operating in one-hot vector space rather than continuous embeddings
[00:25:50] - Evaluating candidate substitutions across all positions to find the best update
[00:28:05] - Running the attack optimization for hundreds of steps across multiple GPUs
[00:39:14] - The difficulty of understanding the loss landscape and internal model workings
[00:43:55] - The flexibility afforded by separating the loss and optimization approach
[00:48:16] - The challenges of creating inherently robust models via adversarial training
[00:52:34] - Potential approaches to defense through filtering or inherent model robustness
[00:55:51] - Transferability results to commercial models like GPT-4 and Claude
[00:59:25] - Hypotheses on why the attacks transfer across different model architectures
[01:04:36] - The mix of human-interpretable and nonsense features in effective attacks
[01:08:29] - The appearance of intuitive manual jailbreak triggers in some attacks
[01:15:33] - Short-term harms of attacks vs long-term risks
[01:18:37] - Influencing those with incomplete understanding of LLMs to appreciate differences from human reasoning
[01:24:16] - Mitigating risks by training on filtered datasets vs broad web data
[01:2916] - Curriculum learning as a strategy for both capability and safety
[01:30:35] - Influencing developers building autonomous systems with LLMs
[01:33:19] - Alienness of LLM failure modes compared to human reasoning
[01:35:45] - Getting inspiration from biological visual system structure
[01:40:35] - Initialization as an alternative to pretraining for small datasets
[01:51:41] - Encoding useful structures like grammars in initialization without training
[02:12:10] - Most ideas don't progress to research projects
[02:13:02] - Pursuing ideas based on interest and feasibility
[02:15:14] - Fun of exploring uncharted territory in ML research
SPONSORS: NetSuite | Omneky
NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.
Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.
Music Credit: Stableaudio.com
The Future of AI Security with Adam Wenchel, CEO of Arthur.ai
Season 1 · Episode 63
Tuesday, September 19, 2023 • Duration 01:23:49
In this episode, Nathan sits down with Adam Wenchel, CEO of Arthur.ai. Adam founded the AI security company back in 2019, before GPT-2 existed. In this episode, Adam shares his unique perspective on the AI security landscape, drawing from years building commercial AI systems. They discuss the attacks Adam set out to defend against, the changing priorities of executives in the rush to adopt LLMs, and the LLM-specific techniques Adam has developed. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive
We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.
TIMESTAMPS:
(00:00:00) Episode Preview
(00:03:45) Adam's background in AI and starting Arthur AI in 2019
(00:05:52) The release of ChatGPT as a watershed moment for generative AI
(00:07:09) Differences between traditional cybersecurity and AI security
(00:09:51) Early examples of AI security issues like boundary detection attacks in fraud systems
(00:12:39) - Mitigating risks of AI systems through observability and robust training
(00:14:40) - Financial services governance of AI models and its challenges today
(00:15:12) Sponsors: Netsuite | Omneky
(00:21:18) - Motivations for governance like staying compliant with regulations
(00:21:40) - The mix of incentives shaping earlier AI governance, like explainability
(00:28:14) - Using LMs to evaluate the security of other LMs
(00:30:03) - Dynamics between training and evaluating future LMs
(00:38:10) - The state of reasoning capabilities in large LMs
(00:44:35) - Corporate urgency around adopting generative AI technologies
(00:46:51) - Common enterprise use cases for generative AI and security concerns
(00:50:45) - Techniques for reducing hallucinations in retrieval augmented LMs
(00:53:15) - Benchmarking LMs on specific organizational tasks versus generic benchmarks
(00:56:30) - Metrics beyond accuracy like concision and hedging
(01:01:20) - Automatically detecting anomalies and hallucinations
(01:09:20) - Relationships between Arthur AI and foundation model providers
(01:11:52) - Where Cohere shines: multilingualism and not hedging
(01:13:43) - Anticipating future watershed moments and steady progress
(01:19:03) - Whether we can ever fully solve AI alignment and safety
LINKS:
Arthur.ai: https://www.arthur.ai/
X/Social:
@apwenchel (Adam)
@itsArthurAI (Arthur.ai)
@labenz (Nathan)
@eriktorenberg
@CogRev_Podcast
SPONSORS: Shopify | NetSuite | Omneky
Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive
NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.
Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.
Music Credit: Stableaudio.com
AI-Powered Filmmaking with Waymark's Stephen Parker and Josh Rubin
Season 1 · Episode 62
Thursday, September 14, 2023 • Duration 01:30:59
In this episode, Nathan sits down with Stephen Parker and Josh Rubin of Waymark, and creators of The Frost, an AI-powered 12 minute short film. In this episode, we get a behind the scenes look at their creative process, the prompting and creative techniques they used to generate and animate the DALL-E results, and an overview of the current state of AI art. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive
We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.
TIMESTAMPS
(00:00) Episode Preview
(00:01:00) Nathan’s introduction for Stephen Parker and Josh Rubin
(00:05:01) - The Frost is a 12-minute short film created using DALL-E 2 images.
(00:07:06) - The Frost started as an experiment to see if a narrative film could be created completely from AI imagery.
(00:10:38) - Parker started generating images with DALL-E 2 when he got access to the early preview.
(00:12:26) - Prompt technique to get consistent images by providing context about a hypothetical film.
(00:15:57) Sponsors: Netsuite | Omneky
(00:19:37) - Compositional continuity, like shot-reverse shot, was hard to achieve through prompting.
(00:22:13)- Rubin would request specific shots and the team would prompt DALL-E 2 to create them
(00:32:25) - Getting consistent facial features for characters was very difficult.
(00:39:03) - The storytelling helped cover inconsistencies that viewers might not notice.
(00:40:15) - Working with the images DALL-E provides
(00:44:53) Inpainting and compositing to refine DALL-E Images
(00:50:43) - The AI art is limited by what exists in the training data.
(01:02:05)- Animating the human characters was challenging because of missing or incorrect appendages.
(01:07:36) - The team had to find creative ways to convey emotion through the limited animation.
01:02:24 - Animating subtle human movement and emotion is still very difficult.
(01:06:35) - A romantic comedy would be much harder to produce with current AI capabilities.
(01:12:17) - For Frost 2 they are using text-to-video models like RunwayML.
(01:15:43) - AI voicing advancements applied to filmmaking
(01:19:27) - The future of AI in Hollywood and filmmaking: quality narratives still require human vision
SPONSORS: Shopify | NetSuite | Omneky
Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive
NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.
Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.
Music Credit: GoogleLM
Related Shows Based on Content Similarities
Discover shows related to "The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis, based on actual content similarities. Explore podcasts with similar topics, themes, and formats, backed by real data.
