Nathan explores the emergence of computational life with Google researchers Ettore Randazzo and Luca Versari. In this episode of The Cognitive Revolution, we delve into their groundbreaking paper on self-replicating programs arising from simple interactions. Join us for a fascinating discussion on the implications for AI development, the origins of life, and the potential future of artificial intelligence. Apply to join over 400 founders and execs in the Turpentine Network: https://hmplogxqz0y.typeform.com/to/JCkphVqj RECOMMENDED PODCAST: 🎙️ This Won't Last - Eavesdrop on Keith Rabois, Kevin Ryan, Logan Bartlett, and Zach Weinberg's monthly backchannel. They unpack their hottest takes on the future of tech, business, venture, investing, and politics. Apple Podcasts: https://podcasts.apple.com/us/podcast/id1765665937 Spotify: https://open.spotify.com/show/2HwSNeVLL1MXy0RjFPyOSz YouTube: https://www.youtube.com/@ThisWontLastpodcast SPONSORS: Building an enterprise-ready SaaS app? WorkOS has got you covered with easy-to-integrate APIs for SAML, SCIM, and more. Join top startups like Vercel, Perplexity, Jasper & Webflow in powering your app with WorkOS. Enjoy a free tier for up to 1M users! Start now at https://bit.ly/WorkOS-Turpentine-Network 80,000 Hours offers free one-on-one career advising for Cognitive Revolution listeners aiming to tackle global challenges, especially in AI. They connect high-potential individuals with experts, opportunities, and personalized career plans to maximize positive impact. Apply for a free call at https://80000hours.org/cognitiverevolution to accelerate your career and contribute to solving pressing AI-related issues. The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/ Head to Squad to access global engineering without the headache and at a fraction of the cost: head to https://choosesquad.com/ and mention “Turpentine” to skip the waitlist. CHAPTERS: (00:00:00) About the Show (00:00:22) Sponsor: WorkOS (00:01:22) About the Episode (00:04:02) Introduction and Paper Overview (00:05:38) Self-Replication and Life (00:07:59) Complexity and Information Theory (00:17:09) Sponsors: 80,000 Hours | Brave (00:19:44) Experiment Setup (00:24:27) Evolution of Self-Replicators (00:34:41) Sponsors: Omneky | Squad (00:36:07) Types of Self-Replicators (00:38:23) Symbiosis and Parasitic Behaviors (00:46:17) Implications for Life in the Universe (01:05:47) Intelligence and Complexity in AI (01:21:03) Concluding Thoughts

Nathan explores the cutting-edge world of AI-powered forecasting with Deger Turan, CEO of Metaculus. In this episode of The Cognitive Revolution, we discuss how AI is revolutionizing prediction markets, the potential for AI to outperform human forecasters, and Metaculus's ambitious new AI forecasting benchmark tournament. Join us for an insightful conversation about the future of decision-making and collective intelligence. Participate in Metaculus' first of its kind bot forecasting tournament: https://www.metaculus.com/aib/ Apply to join over 400 founders and execs in the Turpentine Network: https://hmplogxqz0y.typeform.com/to/JCkphVqj RECOMMENDED PODCAST: 🎙️ This Won't Last - Eavesdrop on Keith Rabois, Kevin Ryan, Logan Bartlett, and Zach Weinberg's monthly backchannel. They unpack their hottest takes on the future of tech, business, venture, investing, and politics. Apple Podcasts: https://podcasts.apple.com/us/podcast/id1765665937 Spotify: https://open.spotify.com/show/2HwSNeVLL1MXy0RjFPyOSz YouTube: https://www.youtube.com/@ThisWontLastpodcast SPONSORS: Building an enterprise-ready SaaS app? WorkOS has got you covered with easy-to-integrate APIs for SAML, SCIM, and more. Join top startups like Vercel, Perplexity, Jasper & Webflow in powering your app with WorkOS. Enjoy a free tier for up to 1M users! Start now at https://bit.ly/WorkOS-Turpentine-Network 80,000 Hours offers free one-on-one career advising for Cognitive Revolution listeners aiming to tackle global challenges, especially in AI. They connect high-potential individuals with experts, opportunities, and personalized career plans to maximize positive impact. Apply for a free call at https://80000hours.org/cognitiverevolution to accelerate your career and contribute to solving pressing AI-related issues. The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/ Head to Squad to access global engineering without the headache and at a fraction of the cost: head to https://choosesquad.com/ and mention “Turpentine” to skip the waitlist. CHAPTERS: (00:00:00) About the Show (00:00:22) Sponsor: WorkOS (00:01:22) About the Episode (00:04:47) Introduction and Background (00:08:42) Deger's Journey to Forecasting (00:13:06) Current State of Forecasting (00:20:23) Sponsors: 80,000 Hours | Brave (00:22:55) Overview of Metaculous (00:23:30) AI Forecasting Research Papers (00:35:44) Sponsors: Omneky | Squad (00:37:31) AI Forecasting Benchmark Tournament (00:44:01) Improving Forecasting Accuracy (00:51:56) Scoring System and Tournament Rules (01:04:17) AI Forecasting Benchmark Series Details (01:11:03) Tournament Structure and Participation (01:17:07) Consistency in AI Forecasting (01:34:36) Risks and Challenges in Forecasting (01:42:43) Consensus Illusion and Policy Making (01:53:55) Outro

Nathan Labenz joins the Nick Halaris Show to discuss the current AI landscape, balancing enthusiasm with critical analysis. This episode of The Cognitive Revolution explores AI's transformative potential in medicine, law, and automation, while addressing concerns about safety, control, and geopolitical implications. Join us for an insightful conversation on responsible AI development and its impact on society. Apply to join over 400 founders and execs in the Turpentine Network: https://hmplogxqz0y.typeform.com/to/JCkphVqj RECOMMENDED PODCAST: 🎙️ Second Opinion - A new podcast for health-tech insiders from Christina Farr of the Second Opinion newsletter. Join Christina Farr, Luba Greenwood, and Ash Zenooz every week as they challenge industry experts with tough questions about the best bets in health-tech. Apple Podcasts: https://podcasts.apple.com/us/podcast/id1759267211 Spotify: https://open.spotify.com/show/0A8NwQE976s32zdBbZw6bv - 🎙️ History 102 with WhatifAltHist Every week, creator of WhatifAltHist Rudyard Lynch and Erik Torenberg cover a major topic in history in depth -- in under an hour. This season will cover classical Greece, early America, the Vikings, medieval Islam, ancient China, the fall of the Roman Empire, and more. Subscribe on Spotify: https://open.spotify.com/show/36Kqo3BMMUBGTDo1IEYihm Apple: https://podcasts.apple.com/us/podcast/history-102-with-whatifalthists-rudyard-lynch-and/id1730633913 YouTube: https://www.youtube.com/@History102-qg5oj SPONSORS: Building an enterprise-ready SaaS app? WorkOS has got you covered with easy-to-integrate APIs for SAML, SCIM, and more. Join top startups like Vercel, Perplexity, Jasper & Webflow in powering your app with WorkOS. Enjoy a free tier for up to 1M users! Start now at https://bit.ly/WorkOS-TCR Oracle Cloud Infrastructure (OCI) is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds; offers one consistent price, and nobody does data better than Oracle. If you want to do more and spend less, take a free test drive of OCI at https://oracle.com/cognitive The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/ Head to Squad to access global engineering without the headache and at a fraction of the cost: head to https://choosesquad.com/ and mention “Turpentine” to skip the waitlist. CHAPTERS: (00:00:00) About the Show (00:00:25) Sponsor: WorkOS (00:01:25) About the Show (00:03:36) Intro (00:05:53) How did you get into AI? (00:11:09) The power of the transformer (00:14:16) Waymark growth (00:19:00) Sponsors: Oracle | Brave (00:21:04) The potential benefits of AI (00:34:20) AI and mass inequality (00:35:26) AI could lead to Universal Basic Income (00:37:37) AI supply and the future of work (00:40:53) The AI future (00:43:38) The new social contract (00:47:06) Optimism about the future (00:53:19) What are people afraid of? (00:58:31) Reinforcement learning from human feedback (01:07:56) What did Ilya see? (01:10:23) Who controls AI? (01:13:23) China (01:16:05) Taiwan and the chip sector (01:20:05) The AI arms race (01:23:22) AI Governance (01:26:28) AI Safety (01:29:22) Custom Models (01:31:33) Closing (01:32:23) Outro

In this episode, Nathan sits down with Swyx of Latent Space to chat about AI engineers and tools to check out, competitive dynamics between OpenAI, other foundation model providers, and developers, and if Swyx would wear an AI Horcrux. We also learn that Nathan lived in the same dorm as Mark Zuckerberg back in his college days (and was a late adopter to Facebook…). If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive Subscribe to the show Substack to join future conversations and submit questions to upcoming guests! https://cognitiverevolution.substack.com/ We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. TIMESTAMPS: (00:00) Episode Preview (00:00:49) AI Nathan’s intro (00:03:14) What is an AI engineer? (00:05:56) What backgrounds do AI engineers typically have? (00:15:51) Sponsors: Netsuite | Omneky (00:17:13) Swyx’s Discord AI project (00:20:41) Key tools for AI engineers (00:23:42) HumanLoop, Guardrails, Langchain (00:27:01) Criteria for identifying capable AI engineers when hiring (00:30:59) Skepticism around AI being a fad and doubts about contributing to AI (00:34:03) AI Engineer Conference speaker lineup (00:41:14) AI agents and two years to AGI (00:46:04) Expectations and disagreement around what AI agent capabilities will work soon (00:50:12) Swyx’s OpenAI thesis (00:53:03) AI safety considerations and the role of AI engineers (00:56:24) Disagreement on whether AI will soon be able to generate code pull requests (01:01:07) AI helping non-technical people to code (01:01:49) Multi-modal Chat-GPT and the future implications (01:03:33) Nathan living in the same dorm as Mark Zuckerberg (01:04:44) Competitive dynamics between OpenAI and other AI model developers (01:05:39) Play.ht vs ElevenLabs (01:09:20) The tension between platforms and developers building on top of them (01:11:40) The best thing startups can do to compete with foundation model providers (01:16:26) User identity/authentication services like Login with OpenAI (01:19:20) Google vs the other live players (01:20:46) AI Horcruxes / Pendants (01:22:05) The concept of an AI app bundle for consumers and developers

In this episode, Aravind Srinivas of Perplexity AI returns to the show. They discuss Perplexity’s growth to millions of queries per day, CEO Aravind’s favourite Perplexity use cases, and how Perplexity ships fast in a competitive landscape against Google and major AI live players creating their own search engines. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. X: @labenz (Nathan) @AravSrinivas (Aravind) @eriktorenberg (Erik) @cogrev_podcast  LINKS: - Perplexity AI: https://www.perplexity.ai/ - Part 1 with Aravind: https://www.youtube.com/watch?v=YSYQZZu4MEM TIMESTAMPS: (00:00:00) - Episode Preview (00:01:12) - Introducing Aravind Srinivas and Perplexity AI (00:05:05) - Perplexity's growth to millions of queries per day (00:06:21) - Perplexity's retention and market share vs. ChatGPT (00:08:32) - Improving reliability, speed, accuracy, and new features (00:10:33) - Getting to daily usage from weekly usage (00:13:21) - Aravind's favorite Perplexity use cases (00:18:34) - Issues with lack of sufficient web coverage for hard queries (00:18:38) - Sponsors: Netsuite | Omneky (00:23:13) - Building their own search index and models for self-sufficiency (00:26:29) - Using their own index vs real-time web queries (00:28:43) - Integrating default search and Copilot companion experiences (00:31:49) - Training their own models as the only viable long-term option (00:36:56) - The web becoming more like an API accessed via AI assistants (00:41:23) - Not trying to copy Google's business model (00:44:36) - Spending more on compute than people to get scale efficiencies (00:49:35) - Replacing all parts of the stack with their own proprietary versions (00:53:59) - Shipping fast out of necessity in a competitive war zone (00:58:00) - AI assistants becoming a significant part of human digital activity The Cognitive Revolution is brought to you by the Turpentine Media network.  Producer: Vivian Meng Executive Producers: Amelia Salyers, and Erik Torenberg Editor: Graham Bessellieu For inquiries about guests or sponsoring the podcast, please email vivian@turpentine.co

Zvi Mowshowitz, the writer behind Don't Worry About the Vase, returns to catch up with Nathan on everything OpenAI, Amazon-Anthropic collab, and Google Deepmind. They also discuss Perplexity, deepfakes, and software bundling vs the Roman Empire. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive Definitely also take a moment to subscribe to Zvi's blog Don't Worry About the Vase  (https://thezvi.wordpress.com/) - Zvi is an information hyperprocessor who synthesizes vast amounts of new and ever-evolving information into extremely clear summaries that help educated people keep up with the latest news.  We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. TIMESTAMPS: 00:00:00 - Episode Preview (00:02:42) - Nathan's experience using Code Interpreter for a React app 00:06:09 - Zvi's perspective on Code Interpreter and other new Anthropic products (00:10:47) - Nathan's approach of "coding by analogy" using Code Interpreter (00:15:42) - Sponsors: Netsuite | Omneky (00:17:00 )- Performance degradation issues with large context windows (00:22:50) - The disconnect between Anthropic's value and what users are willing to pay (00:31:56) - Predicting Gemini's capabilities relative to GPT-4 00:30:13 - Rating Code Interpreter's capabilities (00:37:53) - Using Perplexity vs. Anthropic products for search (00:44:11) - Potential for a bundled subscription for multiple AI services (00:47:39) - Challenges of getting competitors to agree to bundling (00:54:05) - Concerns over information pollution from synthetic content (00:56:36) - Filtering adversarial vs. unintentional bogus information (01:02:20) - Dangers of info pollution visible in Archive dataset (01:11:15) - Kevin Fisher’s AI Souls demo with emotional voices (01:12:15) - Difficulty of detecting AI voices/images for a general audience (01:14:32) - Being optimistic about defending against deepfakes (01:21:12) - The reversal curse in language models (01:46:12) - Implications of Amazon investing in Anthropic (01:49:20) - Non-standard terms likely affected the Anthropic valuation (01:51:13) - Survey of the AI Safety landscape

Nathan and Erik chat OpenAI GPT-3.5 fine tuning updates, using GPT 4 outputs to fine-tune 3.5, when to accelerate, and AI bundles. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. TIMESTAMPS: (00:00) - Intro and GPT-3.5 fine tuning update (05:50) - Using GPT-4 to generate training data for GPT-3.5  (09:54) - Potential for training models on synthetic/cleaned data  (10:24) - Using chain of thought prompts to improve model performance  (15:02) - Sponsors: Netsuite | Omneky (19:39) - Accelerating applications vs new AI models  (20:40) - What to accelerate vs what to slow down (26:07) - AI as a co-pilot vs fully automating tasks  (30:12) - When to delegate to AI (36:40)- Displacement of human roles by AI systems (40:39) - Does training on synthetic data solve a problem? (45:10) - The idea of an AI bundle/subscription  (50:28) - Bundling in other industries like cable and SaaS (54:27) - Churn and retention challenges for AI apps  (01:02:54) - Low retention for easy-to-use AI apps (01:03:57) - Incentives for AI companies to join a bundle (01:04:39) - Potential for collaboration between AI companies  (01:12:01) - Leading AI firms creating separate bundles (01:16:43) - Outro

In this episode, Nathan sits down with three researchers at Carnegie Mellon studying adversarial attacks and mimetic initialization: Zico Kolter, Andy Zou, and Asher Trockman. They discuss: the motivation behind researching universal adversarial attacks on language models, how the attacks work, and the short term harms and long term risks of these jailbreaks. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. TIMESTAMPS: [00:00:00] - Introducing the podcast and guests Zico Kolter, Andy Zou, and Asher Trockman [00:06:32] - Discussing the motivation and high-level strategy for the universal adversarial attack on language models [00:09:33] - Explaining how the attacks work by adding nonsense tokens to maximize target sequence probability [00:11:06] - Comparing to prior adversarial attacks in vision models [00:13:47] - Details on the attack optimization process and discrete token search [00:17:09] - The empirical notion of "mode switching" in the language models [00:21:18] - Technical details on gradient computation across multiple models and prompts [00:23:46] - Operating in one-hot vector space rather than continuous embeddings [00:25:50] - Evaluating candidate substitutions across all positions to find the best update [00:28:05] - Running the attack optimization for hundreds of steps across multiple GPUs [00:39:14] - The difficulty of understanding the loss landscape and internal model workings [00:43:55] - The flexibility afforded by separating the loss and optimization approach [00:48:16] - The challenges of creating inherently robust models via adversarial training [00:52:34] - Potential approaches to defense through filtering or inherent model robustness [00:55:51] - Transferability results to commercial models like GPT-4 and Claude [00:59:25] - Hypotheses on why the attacks transfer across different model architectures [01:04:36] - The mix of human-interpretable and nonsense features in effective attacks [01:08:29] - The appearance of intuitive manual jailbreak triggers in some attacks [01:15:33] - Short-term harms of attacks vs long-term risks [01:18:37] - Influencing those with incomplete understanding of LLMs to appreciate differences from human reasoning [01:24:16] - Mitigating risks by training on filtered datasets vs broad web data [01:2916] - Curriculum learning as a strategy for both capability and safety [01:30:35] - Influencing developers building autonomous systems with LLMs [01:33:19] - Alienness of LLM failure modes compared to human reasoning [01:35:45] - Getting inspiration from biological visual system structure [01:40:35] - Initialization as an alternative to pretraining for small datasets [01:51:41] - Encoding useful structures like grammars in initialization without training [02:12:10] - Most ideas don't progress to research projects [02:13:02] - Pursuing ideas based on interest and feasibility [02:15:14] - Fun of exploring uncharted territory in ML research SPONSORS: NetSuite | Omneky NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. Music Credit: Stableaudio.com

In this episode, Nathan sits down with Adam Wenchel, CEO of Arthur.ai. Adam founded the AI security company back in 2019, before GPT-2 existed. In this episode, Adam shares his unique perspective on the AI security landscape, drawing from years building commercial AI systems. They discuss the attacks Adam set out to defend against, the changing priorities of executives in the rush to adopt LLMs, and the LLM-specific techniques Adam has developed. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. TIMESTAMPS: (00:00:00) Episode Preview (00:03:45) Adam's background in AI and starting Arthur AI in 2019 (00:05:52) The release of ChatGPT as a watershed moment for generative AI (00:07:09) Differences between traditional cybersecurity and AI security (00:09:51) Early examples of AI security issues like boundary detection attacks in fraud systems (00:12:39) - Mitigating risks of AI systems through observability and robust training (00:14:40) - Financial services governance of AI models and its challenges today (00:15:12) Sponsors: Netsuite | Omneky (00:21:18) - Motivations for governance like staying compliant with regulations (00:21:40) - The mix of incentives shaping earlier AI governance, like explainability (00:28:14) - Using LMs to evaluate the security of other LMs (00:30:03) - Dynamics between training and evaluating future LMs (00:38:10) - The state of reasoning capabilities in large LMs (00:44:35) - Corporate urgency around adopting generative AI technologies (00:46:51) - Common enterprise use cases for generative AI and security concerns (00:50:45) - Techniques for reducing hallucinations in retrieval augmented LMs (00:53:15) - Benchmarking LMs on specific organizational tasks versus generic benchmarks (00:56:30) - Metrics beyond accuracy like concision and hedging (01:01:20) - Automatically detecting anomalies and hallucinations (01:09:20) - Relationships between Arthur AI and foundation model providers (01:11:52) - Where Cohere shines: multilingualism and not hedging (01:13:43) - Anticipating future watershed moments and steady progress (01:19:03) - Whether we can ever fully solve AI alignment and safety LINKS: Arthur.ai: https://www.arthur.ai/ X/Social: @apwenchel (Adam) @itsArthurAI (Arthur.ai) @labenz (Nathan) @eriktorenberg @CogRev_Podcast SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. Music Credit: Stableaudio.com

In this episode, Nathan sits down with Stephen Parker and Josh Rubin of Waymark, and creators of The Frost, an AI-powered 12 minute short film. In this episode, we get a behind the scenes look at their creative process, the prompting and creative techniques they used to generate and animate the DALL-E results, and an overview of the current state of AI art. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. TIMESTAMPS (00:00) Episode Preview (00:01:00) Nathan’s introduction for Stephen Parker and Josh Rubin (00:05:01) - The Frost is a 12-minute short film created using DALL-E 2 images. (00:07:06) - The Frost started as an experiment to see if a narrative film could be created completely from AI imagery. (00:10:38) - Parker started generating images with DALL-E 2 when he got access to the early preview. (00:12:26) - Prompt technique to get consistent images by providing context about a hypothetical film. (00:15:57) Sponsors: Netsuite | Omneky (00:19:37) - Compositional continuity, like shot-reverse shot, was hard to achieve through prompting. (00:22:13)- Rubin would request specific shots and the team would prompt DALL-E 2 to create them (00:32:25) - Getting consistent facial features for characters was very difficult. (00:39:03) - The storytelling helped cover inconsistencies that viewers might not notice. (00:40:15) - Working with the images DALL-E provides (00:44:53) Inpainting and compositing to refine DALL-E Images (00:50:43) - The AI art is limited by what exists in the training data. (01:02:05)- Animating the human characters was challenging because of missing or incorrect appendages. (01:07:36) - The team had to find creative ways to convey emotion through the limited animation. 01:02:24 - Animating subtle human movement and emotion is still very difficult. (01:06:35) - A romantic comedy would be much harder to produce with current AI capabilities. (01:12:17) - For Frost 2 they are using text-to-video models like RunwayML. (01:15:43) - AI voicing advancements applied to filmmaking (01:19:27) - The future of AI in Hollywood and filmmaking: quality narratives still require human vision SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. Music Credit: GoogleLM

In this episode, Trey Kollmer, WGA Writer and Co-Executive of the show Ghosts, returns to the show to discuss updates to the Hollywood Strikes, including news on SAG-AFTRA and WGA. Trey and Nathan chat why actors are joining the strikes, how AI will change acting as a profession, Trey’s views on reasoning and how he’s experimenting with GPT-4, and much, much more. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. TIMESTAMPS: (00:04:01) Hollywood and AI: updates on SAG-AFTRA and the WGA (00:15:20) Sponsors: NetSuite | Omneky (00:18:56) Studio approach to copyright and compensation with generative AI (00:22:17) Hollywood receptiveness to using AI and protections guild members are asking for (00:24:56) Models implementing gradient descent in the weights (00:29:14) How Nathan uses models to write for the podcast (00:34:15) Generating and mining jokes in the writer’s room (00:40:18) Untraining models (00:44:20) AI writing tools and writer perception of them (00:46:34) Context length and pooling layers (00:51:11) Microsoft China (01:00:02) Actors’ strike (01:05:45) Using 1 million DALL-E images to create an AI short film (01:09:11) Deepfakes (01:12:51) The future where anyone can be a reasonable synthetic actor (01:17:33) Trey’s take on reasoning and why Hollywood should be more open to AI (01:25:49) A monk’s experience with Chat-GPT (01:28:56) Outdatedness of stochastic parrot notion; reasoning and synthesis (01:36:14) Model vs human susceptibility to adversarial attacks because of human robustness (01:47:30) Performance jumping with abstraction (01:49:22) Language model self-delegation (01:51:04) NVIDIA’s margins compared to TSMC and ASML (02:07:35) Adding an AI layer and competing with incumbents (02:13:10) Rewind AI (02:19:27) Big tech vs old school studios (02:14:52) Dramatic ironies from the picketing lines (02:21:38) AI development moments that feel like a movie (02:26:34) CEO of Inflection’s views on misuse being a greater threat than the AI itself SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. Music Credit: GoogleLM

In this episode, Nathan sits down with three members of the a16z x Convex AI Town project: Yoko Li (Partner, a16z), Martin Casado (GP, a16z), and James Cowling (CTO, Convex). AI Town is a virtual town where AI agents live, interact, and socialize. They discuss how AI Town originated from Yoko’s companion app project, unpredictability as a feature in LLMs and interacting with models like they are lifeforms, and why they chose Javascript and Convex to build AI Town. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. 📣 CALL FOR FEEDBACK: To borrow from a meme… we're in the podcast arena trying stuff. Some will work. Some won't. But we're always learning. http://bit.ly/TCRFeedback Fill out the above form to let us know how we can continue delivering great content to you or sending the feedback on your mind to tcr@turpentine.co.  TIMESTAMPS: (00:00:00) - Episode Preview: Intro to AI Town and the idea of AI as companions (00:05:29) - Overview of AI Town and the simulation framework based on the Stanford Generative Agents paper (00:08:24) - Yoko explains how the idea for AI Town originated from the companion app project (00:12:31) - The simplicity and elegance of the AI Town codebase (00:15:47) - Sponsors: Netsuite | Omneky (00:18:25) - How Convex built a server-side game engine for AI Town  (00:23:39 )- James emphasizes the power of functional programming paradigms like Convex for building AI apps (00:25:02) - Using simple JavaScript so anyone could understand and extend AI Town (00:30:23) - How AI coding assistants were used in building AI Town  (00:31:22) - No open source code for the Stanford paper when they started (00:33:25) - The interplay between programmer and AI model (00:38:01)- Martin draws a distinction between using formal languages vs. natural language (00:43:21) The balance between formal language and unpredictable behaviours in LLMs (00:48:27) Are we living in a simulation? (00:50:38) Advice for other developers in AI (00:54:29) AI Town is a community project to be extended on SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. Music Credit: GoogleLM

In this episode, Nathan sits down with Daniel Kang, Assistant Professor of Computer Science at the University of Illinois. Kang has done pioneering work bringing zero knowledge cryptographic proofs to AI. In this episode, they chat about the cryptographic theory behind Daniel's work, how cryptography allows us to balance the tradeoff between privacy and authenticity, and how cryptography usage is needed in a world where LLMs are increasingly embedded into our daily lives. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. CALL FOR FEEDBACK: To borrow from a meme… we're in the podcast arena trying stuff. Some will work. Some won't. But we're always learning. http://bit.ly/TCRFeedback Fill out the above form to let us know how we can continue delivering great content to you or sending the feedback on your mind to tcr@turpentine.co.  TIMESTAMPS: (00:00) Episode Preview (00:01:04) Nathan's Introduction (00:07:06) Motivation for bringing zero-knowledge proofs to AI (00:10:27) Verifying model execution without revealing model details (00:12:42) Verifying medical AI services haven't been tampered with (00:15:09) Sponsors: Netsuite | Omneky (00:18:54) Cryptographic hashes for commitments (00:22:42) Assumptions underlying cryptographic hashes  (00:25:20) Adding entropy through salting  (00:26:24) Z case snarks and the proving process  (00:31:00) Using lookup tables for nonlinearities (00:33:35) Floating point vs fixed point calculations  (00:34:08) Quantizing models for efficiency (00:35:55) Using polynomials to represent arbitrary computations (00:37:26) What are finite fields?  (00:41:23) Toxic waste for cryptographic secrecy (00:45:51) Computational costs (00:47:39) The experience of using a cryptography application to verify model output   (00:49:05) Verification key doesn't reveal model weights (00:56:36) What using crypto infrastructure in AI enables and challenges to its implementation  (01:01:26) Potential for 10-100x cost reductions (01:04:51) Authenticating images with attested cameras (01:11:56) How cryptography in AI could impact daily life (01:14:25) On-device credential verification (01:15:50) Potential for regulation of hardware authentication (01:18:52) Upcoming work to reduce proof costs LINKS: Daniel's website SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. Music Credit: GoogleLM

Nathan hosts Riley Goodside, the world's first staff prompt engineer at Scale AI, to discuss the evolution of prompt engineering. In this episode of The Cognitive Revolution, we explore how language models have progressed, making prompt engineering more like programming than poetry. Discover insights on enterprise AI applications, best practices for pushing LLMs to their limits, and the future of AI automation. Apply to join over 400 founders and execs in the Turpentine Network: https://hmplogxqz0y.typeform.com/to/JCkphVqj RECOMMENDED PODCAST: 🎙️ Second Opinion - A new podcast for health-tech insiders from Christina Farr of the Second Opinion newsletter. Join Christina Farr, Luba Greenwood, and Ash Zenooz every week as they challenge industry experts with tough questions about the best bets in health-tech. Apple Podcasts: https://podcasts.apple.com/us/podcast/id1759267211 Spotify: https://open.spotify.com/show/0A8NwQE976s32zdBbZw6bv - 🎙️ History 102 with WhatifAltHist Every week, creator of WhatifAltHist Rudyard Lynch and Erik Torenberg cover a major topic in history in depth -- in under an hour. This season will cover classical Greece, early America, the Vikings, medieval Islam, ancient China, the fall of the Roman Empire, and more. Subscribe on Spotify: https://open.spotify.com/show/36Kqo3BMMUBGTDo1IEYihm Apple: https://podcasts.apple.com/us/podcast/history-102-with-whatifalthists-rudyard-lynch-and/id1730633913 YouTube: https://www.youtube.com/@History102-qg5oj SPONSORS: Building an enterprise-ready SaaS app? WorkOS has got you covered with easy-to-integrate APIs for SAML, SCIM, and more. Join top startups like Vercel, Perplexity, Jasper & Webflow in powering your app with WorkOS. Enjoy a free tier for up to 1M users! Start now at https://bit.ly/WorkOS-TCR Oracle Cloud Infrastructure (OCI) is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds; offers one consistent price, and nobody does data better than Oracle. If you want to do more and spend less, take a free test drive of OCI at https://oracle.com/cognitive The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/ Head to Squad to access global engineering without the headache and at a fraction of the cost: head to https://choosesquad.com/ and mention “Turpentine” to skip the waitlist. CHAPTERS: (00:00:00) About the Show (00:00:23) Sponsor: WorkOS (00:01:24) Introduction (00:06:23) LLMs using LLMs (00:09:38) Tool Use (00:11:06) How to manage the breadth of the task (00:14:51) Prompt engineering (00:26:49) Multitasking fine-tuning (00:31:49) Sponsors: Omneky | Squad (00:33:36) Best models for fine-tuning (00:36:41) The Platonic Representation Hypothesis (00:42:02) How close are we to AGI? (00:45:44) How do you know if youre being too ambitious? (00:51:18) Best practices for generating good output (00:54:33) Backfills and synthetic transformations (00:56:59) Prompt engineering (01:05:54) AGI, modalities, and the limits of training (01:20:08) Solving the ARC Challenge (01:23:20) How to Demonstrate Prompt Engineering Skills (01:25:27) Outro

In this episode, Nathan and Erik sit down to analyze Hugging Face in light of its recent $235M Series D round. They analyze Hugging Face’s community and defensibility through the lens of other community businesses like ProductHunt and Yelp, assess its ability to fulfill its $4.5 billion valuation, and assess competitors and other notable companies in the space like Replit, Character, and Runway. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. TIMESTAMPS: (00:00:57) Episode Preview (00:01:52) Nathan’s Introduction (00:04:57) Overview of Hugging Face and recent fundraising announcement (00:08:13) Hugging Face’s product line (00:16:47) Sponsors: Netsuite | Omneky (00:18:36) Community driven businesses and HuggingFace’s moat (00:23:23) Discovery and inference (00:28:04) Hugging Face’s ideological nature (00:31:31) Curation is key (00:33:46) Keeping content fresh when AI moves so fast (00:35:44) If Hugging Face is a $50 billion company one day, what would it look like? (00:48:08) Hugging Face vs Replit (00:57:19) Contrasting Hugging Face with other companies like Character and Runway LINKS MENTIONED: Hugging Face's LLM leaderboard X/TWITTER: @labenz (Nathan) @eriktorenberg @CogRev_Podcast SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. Music Credit: GoogleLM

In this episode, Nathan sits down with Paige Bailey, Lead Product Manager of Generative Models at Google Deepmind. In this conversation, they discuss what it's like to be a PM for an LLM as opposed to an app, defining ideal LLM behaviour, and reasoning - how do you distinguish real abilities vs pattern matching? If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. TIMESTAMPS: (00:00) Episode Preview (00:01:15) Introducing Paige Bailey (00:04:21) Paige’s background at Google Brain and the Deepmind merger (00:07:00) PM for a LLM vs being a PM for an app (00:11:21) The development timeline and compute budget of PaLM-2 (00:14:30) Paige’s role in the PaLM 2 project (00:15:30) Sponsors: Netsuite | Omneky (00:17:26) Defining desired capabilities for PaLM-2 (00:19:17) The amount of work that went into elevating PaLM 2 from PaLM 1 (00:20:28) Has Google lost its ability to ship? (00:24:240) Paige's "eureka" moment seeing GitHub Copilot capabilities (00:27:47) Competing PaLM 2 with other models (00:32:20) Grokking and the predictability of emergent capabilities (00:37:30) Citizen scientists and the multilingual capabilities of PaLM 2 (00:39:29) Distinguishing real reasoning vs pattern matching (00:45:51) Products using PaLM-2 that people should try (00:50:35) Most exciting AI projects that you can try out (00:52:29) Curriculum learning and successor to the transformer LINKS: PaLM 2 Duet AI for developers Avenging Polayni’s Revenge X/TWITTER: @DynamicWebPaige (Paige) @labenz (Nathan) @eriktorenberg @CogRev_Podcast SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. Music Credit: GoogleLM

Join Nathan Labenz and Erik Torenberg as they analyze the latest developments from OpenAI on GPT 3.5, compare GPT to other live player models like Llama2, and discuss the state of AI in coding, education, and healthcare. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics TIMESTAMPS: (00:00) Episode Preview (00:01:00) GPT 3.5 Turbo (00:06:36) Llama 2 vs GPT (00:11:24) How much inference is needed to double compute costs? (00:13:40) OpenAI’s moat (00:14:41) OpenAI’s privacy consideration for data (00:16:06) Sponsor: Netsuite | Omneky (00:17:46) Encouraging the usage of instructions during fine-tuning (00:19:19) Live player consideration of AI safety (00:22:35) Da Vinci: new completions fine tuneable model (00:24:59) Chat-GPT usage in decline (00:30:03) Getting on demand tutoring on ML papers (00:31:00) Code, education, and healthcare (00:31:42) AI applications in coding (00:38:12) AI revolution n education (00:42:35) AI revolution in healthcare (00:52:17) Call for feedback LINKS: Replit episode with Tyler Angert Replit episode with VP of AI, Michele Catasta AI Revolution in Education with Khan Academy's Director of Engineering, Shawn Jansepar Google's Multimodal Med-PaLM with Vivek Natarajan and Tao Tu X: @labenz (Nathan) @eriktorenberg (Erik) @cogrev_podcast SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

In this episode, Nathan sits down with Vivek Natarajan and Tao Tu of Google’s Med-PaLM, diving into how they used one of the world’s largest medical datasets ever compiled to develop Med-PaLM M, an AI agent specialized in medical tasks. In this episode, they discuss: Med-PaLM M's “clinically superhuman” abilities and limitations, the rigorous testing and validation that went into the model, and their vision for AI to take over repetitive clerical tasks and allow doctors to focus on patients. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. TIMESTAMPS: (00:00) Episode Preview (00:00:56) Introducing Vivek Natarajan and Tao Tu (00:04:18) The story of Google’s Medical AI research progress (00:07:11) Multi-modal Med-PaLM (00:10:32) Genomic data - how do you represent it? (00:11:13) Google’s Deep Variant (00:14:44) The successes and failures behind the incredible pace of progress (00:15:02) Sponsors: Netsuite | Omneky (00:21:54) Google’s research culture and assembling an interdisciplinary team (00:31:36) Google’s Pathways (00:33:40) Med-PaLM M's architecture (00:37:28) Working with 3 different model sizes and what you learn (00:46:56) Data and compute required for Med-PaLM M (00:49:38) Med-PaLM M's cycle time (00:54:56) Is a bridge or adapter structure worth implementing? (01:00:09) Can we create an AI doctor? (01:02:39) Emergent capabilities like identifying tuberculosis (01:09:37) Reactions to these emergent capabilities (01:11:13) Moving towards clinical trials and real-world testing (01:13:01) Regulatory and safety considerations (01:15:03) AI safety in the healthcare domain (01:17:00) Potential to transform healthcare access worldwide LINKS: Med-PaLM: https://sites.research.google/med-palm/ Med-PaLM M paper: https://arxiv.org/abs/2307.14334 Our earlier conversation with Vivek Natarajan on Med-PaLM: https://www.youtube.com/watch?v=nPBd7i5tnEE X/TWITTER: @vivnat (Vivek) @taotu831 (Tao) @labenz (Nathan) @eriktorenberg @CogRev_Podcast SPONSORS: NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

In this episode, Nathan sits down with Jon Noronha, co-founder of Gamma. Gamma is a new medium for presenting ideas, allowing you to focus on your ideas and receive beautiful, engaging content without the formatting work. In this episode, Jon and Nathan discuss the journey of building Gamma, how to coax AI to do things well, and the opportunity for AI A/B testing. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. TIMESTAMPS: (00:00:00) - Preview (00:01:23) - Nathan introduces Jon Noronha (00:06:36) - Intro - Jon Noronha introduces himself and Gamma (00:09:36) - Gamma's origin story: building the "anti-PowerPoint" pre-GPT-3 (00:13:15) - From using AI for onboarding to generating near-complete presentations. (00:15:11) - Sponsors: Netsuite | Omneky (00:18:01) - If Notion and Canva had a baby (00:23:35) - Searching for the right structure for AI, landing on HTML (00:28:16) - Degrees of freedom - developing a constrained vocabulary of semantic blocks (00:29:30) - Choice of model(s) – weighing cost, speed, and quality tradeoffs (00:35:18) - Editing UI - getting AI to edit slides per user command (00:38:04) - UI for gathering training data - following Midjourney's example (00:40:00) AI A/B Testing: a big opportunity space (00:43:22) - Image generation models - Baseten and Gamma's deployment of it (00:45:37) - Pacing feature development - Trying to stay ahead of innovations like 3D, video, voice etc. (00:47:45) Image generation for slidedecks and AI involvement (00:50:29) Gamma's main building priorities and AI generated websites (00:52:34) - Competing with giants like Microsoft and Google (00:55:04) - Enterprise AI shipping challenges - Startups pioneer new patterns; giants will eventually follow. (00:57:45) - Simplifying complexity - will models smooth over the complexity of bloated product suites? (00:59:22) - Conclusion and final thoughts X/TWITTER: @thatsjonsense (Jon) @labenz (Nathan) @eriktorenberg @CogRev_Podcast LINKS: Gamma: https://gamma.app/ SPONSORS: Shopify |NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

In this episode, Nathan sits down with Shawn Jansepar, Director of Engineering at Khan Academy, to discuss their GPT-4 powered Socratic tutor, Khanmigo. In this conversation, Shawn and Nathan chat about Khan Academy’s collaboration with OpenAI and how they helped fine-tune GPT-4, how Khan Academy leveraged GPT-4 to build Khanmigo, and the impact of providing access to an AI tutor to any student. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. TIMESTAMPS: (00:00) Episode Preview: Education 10 years from now (04:42) Khan Academy’s early access partnership with OpenAI (06:31) Khanmigo: journey from Chrome extension to AI tutor (11:36) GPT-4’s ability to be Socratic vs 3.5 (15:05) Sponsors: Netsuite | Omneky (16:40) Integrating Khan Academy’s Pedagogy into AI (20:06) The future of education 10 years from now  (22:37) Khan Academy’s models (27:20) Demo-driven development process (31:16) Sculpting the behaviour of a Socratic tutor model  (35:59) Khan Academy’s contribution to GPT-4’s fine-tuning and RLHF (38:41) Being data-informed vs data-driven as a practice  (42:10) Incurring tech debt to get ahead of the curve (45:28) The boundary for what an AI can/can’t t tutor  (49:30) Identifying when the user is confused and avoiding AI hallucinations (53:54) Khanmigo’s development patterns  (59:11) Making Khanmigo jailbreak resistant  (01:01:50) Delivering personalized education with AI  (01:04:08) How Shawn and his team are thinking about AI education (01:05:33) Khanmigo’s future multimodal interactivity (01:08:42) Evaluating Khanmigo’s efficacy for student learning  (01:11:41) How widely is Khanmigo deployed today and what is the future for universal public access? (01:05:11) Distribution through teachers and districts (01:16:30) What are the reactions from teachers and education institutions to AI? (01:18:15) Khanmigo’s pricing model (01:19:03) The future roadmap for Khanmigo (01:20:45) How will the AI tutor change the world at large? LINKS: Khanmigo: khan.co/khanmigo23 Benjamin Bloom’s 2-Sigma Problem: https://web.mit.edu/5.95/www/readings/bloom-two-sigma.pdf X/TWITTER: @shawnjan8 (Shawn) @labenz (Nathan) @eriktorenberg @CogRev_Podcast SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

Join Nathan Labenz and Erik Torenberg as they analyze the last month in AI advancements. Nathan takes us through the meaningful updates to his Scouting Report (released last month, linked below), discusses highlights from recent episodes of The Cognitive Revolution, and gives us a sneak peek at upcoming interviews with Google researchers. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. TIMESTAMPS: (01:00) How does the AI Scouting Report hold up a few weeks later? (03:29) Zvi’s feedback on Nathan’s Tale of the Cognitive Tape (10:25) The universal LLM jailbreak and adversarial examples (12:53) Human performance is much more variable than AI performance (14:45) Sponsors: NetSuite | Omneky (16:09) Nathan’s AI Task Automation: What are good targets for tasks that can be automated for average businesses?  (20:05) Is GPT-4 getting worse or better? (22:00) Getting explicit about what good looks like (24:00) Prompting best practices are very accessible (26:35) Ghostwriting - and the art of the hook (28:05) Live Players: Which companies have say so over how the future goes? (31:10) Upcoming guests from Google AI (35:40) Possible post-transformer architectures LINKS: SCOUTING REPORT Part 1: https://youtu.be/0hvtiVQ_LqQ SCOUTING REPORT Part 2: https://youtu.be/ovm4MbQ4G9E SCOUTING REPORT Part 3: https://youtu.be/QJi0UJ_DV3E 3 Blue 1 Brown on YouTube: https://www.youtube.com/@3blue1brown Tale of the Cognitive Tape in Part 1 of the Scouting Report: https://www.youtube.com/watch?v=0hvtiVQ_LqQ&t=3043s Analyzing the Frontier with Zvi Mowshowitz: https://www.youtube.com/watch?v=SM4q-QAsoU8&t=1s Tyler Cowen’s Interview with Jonathan Swift: https://conversationswithtyler.com/episodes/jonathan-gpt-swift/ X: @labenz (Nathan) @eriktorenberg (Erik) @cogrev_podcast SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.  Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

In this episode, Nathan sits down with Replit’s VP of AI, Michele Catasta. Replit is building what CEO Amjad Masad calls "the perfect substrate for AGI." In this discussion, Michele and Nathan discuss Replit's state of AI development report, advantages when it comes to AI development, and the company's custom models. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. TIMESTAMPS: (00:00) Episode Preview (00:00:57) Introduction (04:44) What is artificial developer intelligence? (15:05) Sponsors: Netsuite | Omneky (16:58) Michele's background at Google & decision to join Replit (19:16) Startups vs incumbents (24:42) Whether Replit identifies as an e/acc company (26:37) Staying apolitical on AI while being responsible (30:36) The rise of LangChain and polarized reactions (35:14) Estimates on developer productivity gains from AI assistance (2x-10x) (38:35) Democratizing software development through easy customization (44:02) AI generating disposable single-use software (51:33) Optimism about humanity's ability to handle transformative AI (55:01) The need for nuanced AI safety discussions (56:14) Replit's data advantage from user code execution (01:04:31) Replit's approach to training custom AI models (01:08:51) The value of both open source and commercial models (01:13:18) Michele’s highlights from being a Google researcher (01:15:47) World knowledge needs in AI development (01:20:37) Replit’s approach to AI safety (01:24:28) The advantage of having a commercial model (01:25:54) The costs of serving AI features to millions of users (01:28:49) Modeling cost per user with AI workloads (01:30:50) Pushing AI inference to the edge (01:32:18) Ghostwriter integrating more deeply into Replit's IDE (01:33:37) Replit as a potential "substrate for AGI" LINKS: https://replit.com/ Replit's State of AI Development Report: https://blog.replit.com/ai-on-replit X: @pirroh (Michele) @labenz (Nathan) @eriktorenberg (Erik) @cogrev_podcast SPONSORS: Shopify | NetSuite | Omneky - Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive - NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you’re looking for an ERP platform, head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. - Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. MUSIC CREDIT: MusicLM

In this episode, Nathan sits down with Tyler Angert, Product Designer at Replit, to discuss the future of software development. Replit is building what CEO Amjad Masad calls "the perfect substrate for AGI." In this discussion, Tyler and Nathan discuss how Replit is leveraging AI to enhance its current product, bot-bot interactions, the design and ethical considerations around AI agents, and more. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. TIMESTAMPS: (00:00) Episode Preview (01:10) Nathan’s Intro (09:24) Tyler's role at Replit (11:37) Replit as the "perfect substrate for AGI" (14:55) Sponsor: Netsuite | Omneky (15:45) Defining AGI (19:26) How AI agents might interact with Replit (23:20) Replit's "virtual developer" product (31:11) Current state of Replit's AI features like Ghostwriter (41:48) Measuring productivity boost from AI coding tools (45:29) Potential for massive developer productivity gains from AI (48:34) Technical gaps still remaining to achieve advanced AI agents (51:45) Core AI breakthroughs have already occurred (57:35) Timeline for functional AI agents interacting online (01:01:20) Safety considerations for powerful AI agents on Replit (01:08:47) Ethical considerations around AI and consciousness (01:20:36) What constitutes consciousness in an AI system (01:26:30) Should AI systems have rights and protections? (01:30:32) Being polite to AI systems (01:31:59) Advice for beginners looking to leverage AI (01:36:58) Using AI with a Neuralink brain implant (01:43:02) Hopes and fears about AI's impact on society LINKS: https://replit.com/ X: @tylerangert (Tyler) @labenz (Nathan) @eriktorenberg (Erik) @cogrev_podcast SPONSORS: Shopify | NetSuite | Omneky  Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive -NetSuite provides financial software for all your business needs. More than thirty-six thousand companies have already upgraded to NetSuite, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you’re looking for an ERP platform: NetSuite (http://netsuite.com/cognitive) and defer payments of a FULL NetSuite implementation for six months.  -Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. MUSIC CREDIT: MusicLM  This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co.

This isn't news, it's analysis! Nathan Labenz sits down for an with Zvi Mowshowitz, the writer behind Don't Worry About the Vase to talk about the major players in AI over the last few months. In this extended conversation, Nathan and Zvi debate if AI has attained the intelligence of a well-read college graduate (per OpenAI's Jan Leike), a live player analysis (who to count/ who not to count), and the role of independent red teaming organizations. If you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive Definitely also take a moment to subscribe to Zvi's blog Don't Worry About the Vase (https://thezvi.wordpress.com/) - Zvi is an information hyperprocessor who synthesizes vast amounts of new and ever-evolving information into extremely clear summaries that help educated people keep up with the latest news. Highly recommend. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. TIMESTAMPS: (00:00) Episode preview (03:15) Is AI as intelligent as a college grad? (07:45) Memories and context processing (15:45) Sponsor: NetSuite | Omneky (17:13) Is AI as intelligent as a college grad? cont'd (20:47) Strengths and weaknesses of AI vs human (31:05) OpenAI Superalignment (37:23) The relationship between OpenAI and Anthropic (44:31) Anthropic's security recommendations and adversarial attacks (50:50) Is OpenAI using a constitutional AI approach?  (01:01:26) Context and stochastic parrots (01:10) Is more context better? (01:15:29) Should Nathan work at Anthropic? (01:21:35) Google DeepMind's RT-2 (01:27:47) Multi-modal Med-PaLM (01:31:50) Speculating about Gato (01:35:10) Skepticism about Med-PaLM usage in radiology (01:41:37) Llama 2 - what is going on at Meta?? (01:51:14) Llama 2 vs other models (01:55:29) Who are the live players? (02:01:38) China's AI developments (02:02:41) Character AI and Inflection (02:05:26) Replit as the perfect substrate for AGI (02:10) AI girlfriends (02:18:53) AI safety: The White House (02:25:43) Bottlenecks to progress (02:35:27) Can new players influence AI policy? (02:39:00) Liabilities (02:47:54) Independent red teaming organizations (02:57:18) Mechanistic interpretability X: @labenz (Nathan) @thezvi (Zvi) @eriktorenberg (Erik) @cogrev_podcast  SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive -NetSuite provides financial software for all your business needs. More than thirty-six thousand companies have already upgraded to NetSuite, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you’re looking for an ERP platform: NetSuite (http://netsuite.com/cognitive) and defer payments of a FULL NetSuite implementation for six months. -Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that *actually work* customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. MUSIC CREDIT: MusicLM

Nathan interviews Bernal Jimenez Gutierrez, creator of HippoRAG, a novel approach to retrieval augmented generation inspired by the human hippocampus. In this episode of The Cognitive Revolution, we explore how HippoRAG improves on traditional RAG systems, its neuroanatomical inspiration, and its potential for handling complex queries requiring multi-hop reasoning. Join us for an insightful discussion on the future of AI memory systems. Apply to join over 400 founders and execs in the Turpentine Network: https://hmplogxqz0y.typeform.com/to/JCkphVqj RECOMMENDED PODCAST: 🎙️ Second Opinion - A new podcast for health-tech insiders from Christina Farr of the Second Opinion newsletter. Join Christina Farr, Luba Greenwood, and Ash Zenooz every week as they challenge industry experts with tough questions about the best bets in health-tech. Apple Podcasts: https://podcasts.apple.com/us/podcast/id1759267211 Spotify: https://open.spotify.com/show/0A8NwQE976s32zdBbZw6bv - 🎙️ History 102 with WhatifAltHist Every week, creator of WhatifAltHist Rudyard Lynch and Erik Torenberg cover a major topic in history in depth -- in under an hour. This season will cover classical Greece, early America, the Vikings, medieval Islam, ancient China, the fall of the Roman Empire, and more. Subscribe on Spotify: https://open.spotify.com/show/36Kqo3BMMUBGTDo1IEYihm Apple: https://podcasts.apple.com/us/podcast/history-102-with-whatifalthists-rudyard-lynch-and/id1730633913 YouTube: https://www.youtube.com/@History102-qg5oj SPONSORS: Oracle Cloud Infrastructure (OCI) is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds; offers one consistent price, and nobody does data better than Oracle. If you want to do more and spend less, take a free test drive of OCI at https://oracle.com/cognitiveThe Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/ Head to Squad to access global engineering without the headache and at a fraction of the cost: head to https://choosesquad.com/ and mention “Turpentine” to skip the waitlist. CHAPTERS: (00:00:00) About the Show (00:02:52) Intro (00:05:10) RAG (00:09:29) Hippocampal Memory Indexing Theory (00:14:29) Neocortex, Parahippocampal Regions, Hippocampus (00:18:35) Hipporag (Part 1) (00:21:12) Sponsors: Oracle | Brave (00:23:20) Hipporag (Part 2) (00:23:29) Understanding the Hippocampus (00:29:37) RAG (00:31:38) Runtime (Part 1) (00:36:30) Sponsors: Omneky | Squad (00:38:17) Runtime (Part 2) (00:38:17) PageRank (00:39:29) Headline Results (00:43:22) Gaia Benchmark (00:46:33) Pathfinding vs Path Following (00:51:24) Future work (00:53:26) Starting with a query (00:58:30) Long context LLMs (01:01:50) Hybrid approach (01:05:03) AI Town (01:06:58) Flair (01:09:05) Getting it right, not cheap (01:12:33) Technologies to highlight (01:16:57) AI capabilities that are still unsolved (01:19:47) Transformers meet neural algorithmic reasoners (01:21:02) Closing (01:21:31) Outro

Arthur Conmy sits down with Nathan Labenz for an accessible deep dive into the state of interpretability research online today. They discuss how pioneering researchers have painstakingly worked to isolate the sub-circuits within transformers that are responsible for different aspects of AI capabilities. Arthur also introduces us to a new ACDC approach that he and his co-authors have taken to automating some of the most time-consuming parts of this work. If you’re looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics The Cognitive Revolution is a part of the Turpentine podcast network. Learn more: Turpentine.co TIMESTAMPS: (00:00) Episode Preview (04:40) What attracted Arthur to mechanistic interpretability? (07:49) LLM information processing: General Understanding vs Stochastic Parrot Paradigm (14:00) ACDC paper: https://arxiv.org/abs/2304.14997  (14:45) Sponsors: NetSuite | Omneky (24:30) Putting together data sets  (32:39) How to intervene in LLMs network activity (36:00) Defining metrics to evaluate the production of correct completions (44:20) The future of the mechanistic interpretability research  (50:00) Extracting upstream activations in the ACDC project and evaluating impact on downstream components. (56:00) Anthropic research findings  (01:08:00) 3-Step process of the ACDC approach (01:22:00) Setting a threshold and validation (01:27:00) Goal of the approach (01:32:00) Compute requirements Correction: at (01:33:00), Arthur meant to say = "quadratic in nodes" (01:35:30) Scaling laws for mechanistic interpretability (01:40:00) Accessibility of this research for casual enthusiasts (01:46:00) Emergence discourse (01:56:00) Path to AI safety LINKS: https://arthurconmy.github.io/ https://arxiv.org/abs/2304.14997  X: @labenz (Nathan) @arthurconmy (Arthur) @cogrev_podcast SPONSORS: NetSuite | Omneky -NetSuite provides financial software for all your business needs. More than thirty-six thousand companies have already upgraded to NetSuite, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you’re looking for an ERP platform: NetSuite (http://netsuite.com/cognitive) and defer payments of a FULL NetSuite implementation for six months. -Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that *actually work* customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co.

Nathan Labenz sits down with Dr. Ronen Dar, CTO and co-founder of Run:ai, an Israel-based company that helps enterprises train and deploy AI models by optimizing GPU usage. The discussion covers how chip makers can meet the soaring demands, geopolitical fears, to the best practices companies can secure compute capacity. If you’re looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics RECOMMENDATION: The AI Scouting Report Playlist Parts 1-3: https://www.youtube.com/watch?v=0hvtiVQ_LqQ&list=PLVfJCYRuaJIXooK_KWju5djdVmEpH81ee TIMESTAMPS: (00:00) Episode Preview (00:51) Introduction to Dr. Ronen Dar (03:30) Run:ai's technology and what differentiates it from other solutions (06:00) Today's market compared to when Dr. Ronen started five years ago (13:40)Run:ai on market competitors like mosaicML (14:55) Sponsors: NetSuite | Omneky (22:00) The process and best practices by which companies secure compute capacity (25:00) Dr. Ronen explains the GPU shortage (31:50) GPU solutions (36:00) Relative pricing across major providers (41:00) What other chip makers are going to be relevant? (49:00) Global outlook for chip production (52:45) Worldview around the US-China AI race (58:00) Can controls on hardware actually control access to AI? LINKS: https://www.run.ai/ SOCIAL MEDIA: @labenz (Nathan) @ronen_dar @runailabs (Run:ai) @cogrev_podcast SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive -NetSuite provides financial software for all your business needs. More than thirty-six thousand companies have already upgraded to NetSuite, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you’re looking for an ERP platform -> NetSuite: http://netsuite.com/cognitive and defer payments of a FULL NetSuite implementation for six months. -Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that *actually work* customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

Head over to YouTube to watch Part 2 of The AI Scouting Report (https://www.youtube.com/watch?v=ovm4MbQ4G9E), supported by slides and visual aides. In Part 2, Nathan Labenz builds on Part 1: AI Fundamental and delves into recent trends and practical applications for AI. Nathan's aim is to impart the equivalent of a high school AP course understanding to listeners in 90-minute installments. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics Listener comments on Part 1: "Incredible summary! I’ve watched probably 20 videos like this and I’ve always been left with unanswered questions." "This is a really excellent overview! Enjoyed going through this and looking forward to episode 2. Thank you for sharing it on here for free!" "Best 90 minutes this old PhD physicist has spent in a long time!" LINKS: Part 1: https://youtu.be/0hvtiVQ_LqQ Part 2: https://www.youtube.com/watch?v=ovm4MbQ4G9E Part 3 released later this week! Questions or topics you want us to review on the podcast? Email TCR@turpentine.co The Cognitive Revolution is a part of the Turpentine podcast network. Learn more: Turpentine.co TWITTER: @CogRev_Podcast @labenz (Nathan) @eriktorenberg (Erik) SPONSOR: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co.

Nathan Labenz interviews Div Garg, founder of MULTI·ON, the world's first personal AI agent and life copilot. Div talks about the product strategy and roadmap for the MULTI·ON browser, their natural language approach to skills, and the steps they are taking to ensure user safety. Div explains how the platform uses a critic model to detect the success or failure of tasks, and how it can be used to book flights, order food, and more. Div also talks about the future of memory systems, such as the user profile feature, and how it can be used to improve the user experience. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics The Cognitive Revolution is part of the Turpentine podcast network. Learn more at turpentine.co Have your AI questions answered on an episode by emailing TCR@turpentine.co Send your friends our 90 min AI Scouting Report with visual aides! The AI Scouting Report Part 1: The Fundamentals is on YouTube at https://www.youtube.com/watch?v=0hvtiVQ_LqQ . Part II is coming next week. TIMESTAMPS: (00:00) Episode preview (06:42) AI agents applied to everyday life. (12:03) AI-driven automation with browser extension. (15:02) Sponsor: Omneky (18:17) AI-driven automation of web tasks. (23:57) Task automation and planning. (29:43) Automate task completion with user validation. (34:45) Lifelong learning agent with high-level skills (40:19) Guiding users to create skills safely. (46:53) AI assistant to simplify lives. (53:14) Unlock parallelism with AI agents. LINKS: MULTI·ON: https://www.multion.ai/ Div Garg; https://divyanshgarg.com/ TWITTER:  @labenz (Nathan)  @DivGarg9 @eriktorenberg (Erik)  @cogrev_podcast SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky (www.omneky.com) for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. Music: GoogleLM

Nathan Labenz sits down with Zvi Mowshowitz, the writer behind Don't Worry About the Vase. Zvi is an information hyperprocessor who synthesizes vast amounts of new and ever-evolving information into extremely clear summaries that help educated people keep up with the latest news. In this episode, we cover his AI safety worldview, an overview of the AI discourse, and who really matters in the AI debates. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics Do you have questions you want us to answer, topic requests, or guest suggestions for upcoming episodes? Email us at TCR@turpentine.co Also, be sure to check out our recent release on YouTube: The AI Scouting Report Part 1: The Fundamentals TIMESTAMPS: (00:00) Episode Preview (05:00) Zvi’s Introduction to AI (07:04) Weekly 10,000+ words / Weekly newsletter (12:34) Language models (18:25) AI Worldview (27:30) Probably of Due (33:10) Inspirations for Content (39:00) Audience for Writings (45:25) Impactful figures’ impact (48:55) Path of the river (55:39) Different camps in AI discourse (01:13:55) Acceleration Front Argument (01:20:08) Large Language Models Today (01:27:00) Spendings in AI (01:36:03) Principles / Virtue Ethicism (01:43:30) Human vs Non-human Universe (01:47:32) AI Safety & “Doomers” (02:02:10) Expectations of Human/AI Relationship (02:10:30) Future of Online Laws and Ethics (02:19:10) What do we do next? (02:34:50) Sources for learning (02:42:08) Conclusion LINKS: Don't Worry About the Vase: https://thezvi.substack.com/ TWITTER: @labenz (Nathan) @theZvi (Zvi) @eriktorenberg (Erik) @cogrev_podcast SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky (www.omneky.com) for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. MUSIC CREDIT: MusicLM

Head to a special YouTube presentation with visual aides (https://youtu.be/0hvtiVQ_LqQ), where Nathan Labenz synthesizes the core fundamentals of AI as simply, clearly, and literally as possible. Nathan's aim is to impart the equivalent of a high school AP course understanding to listeners in 90 minutes. This special episode is in part brought to you by Athena. You can use Nathan’s referral link (http://athenago.me/nathan-labenz) for a free first month of Athena services for qualified customers. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics Questions or topics you want us to review for Part 2? Email TCR@turpentine.co The Cognitive Revolution is a part of the Turpentine podcast network. Learn more: Turpentine.co TWITTER: @CogRev_Podcast @labenz (Nathan) @eriktorenberg (Erik) SPONSOR: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co.

In this episode, Nathan Labenz sits down with Hayk Martiros, VP of Autonomy at Skydio. Martiros leads the autonomy team, which holds the state of the art in complex autonomous flight. Skydio is the largest drone developer in the US, and recently raised $230M to expand its US manufacturing 10-fold, bringing its valuation to $2.2 billion. In this discussion, Hayk and Nathan discuss why Skydio will not weaponize its drones, Skydio’s systems and AI approach, and the considerations of choosing an autonomous system over a human agent.  We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. LINKS: SymForce: https://github.com/symforce-org/symforce End-to-End Learning of Geometry and Context for Deep Stereo Regression (Skydio’s 2017 paper): https://arxiv.org/pdf/1703.04309 TIMESTAMPS: (00:00) Episode Preview (01:12) Nathan’s episode intro (07:44) Hayk’s story (11:21) What is Skydio? (14:59) Sponsor: Omneky (16:56) The core driver of Skydio’s business (20:21) Skydio’s AI approach (23:56) Updating the voltage 30,000 times per second (24:52) How is the Skydio system built? (34:07) The loops in the Skydio system (36:00) SymForce, Skydio’s open source library and Hayk’s creation (41:06) Writing symbolic expressions as a roboticist  (42:14) Writing an expression for a toy race car (47:58) Skydio neural networks (51:29) Going from designed systems to end-to-end systems (58:17) How much does a Skydio drone and dock cost to install? (01:00:19) The cost savings of using a Skydio drone (01:02:59) Are $1 drone missions in the consumer’s future? (01:06:22) Vertical integration vs platform (01:09:29) The advantages and disadvantages of drones (01:11:17) Running LLMs on a drone (01:14:36) Power consumption going to compute vs mechanics (01:18:26) Hayk’s predictions for consumer autonomous systems (01:21:51) Standards for choosing autonomous systems over humans (01:25:23) Skydio’s societal impact (01:28:34) Universal basic income (01:30:27) Skydio drone usage in defense (01:34:34) The Ukraine-Russia conflict (01:38:02) Hayk’s experience with the Nagorno-Karabakh conflict TWITTER: @SkydioHQ (Skydio) @CogRev_Podcast @labenz (Nathan) @eriktorenberg (Erik) SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co. MUSIC CREDIT: MusicLM

[Bonus Episode] Latent.space hosts, Alessio and Swyx, sit down with Linus Lee of Notion AI to discuss Linus’ experience starting the AI/UX community, prompt engineering at Notion, and designing AI interfaces and agents.  The latent.space podcast aims to be the first place where AI engineers hear about the latest AI news and technology trends. We’ve had several guests in common – including Shreya Rajpal of Guardrails, Jonathan & Abhi from the recently acquired for $1.3B MosaicML, and also Riley Goodside from Scale AI – and also explored similar themes, including the question of which AI companies have durable business moats, and the flurry of experimentation going into AI agents right now. We recommend checking them out!  Keep up to date with Latent.space here:  Website: https://www.latent.space/ Spotify: https://open.spotify.com/show/2p7zZVwVF6Yk0Zsb4QmT7t?si=95fb033e3c014975 Apple Podcasts: https://podcasts.apple.com/us/podcast/latent-space-the-ai-engineer-podcast-codegen-agents/id1674008350 Youtube: https://www.youtube.com/@LatentSpace-podcast We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics TIMESTAMPS (05:52) Starting the AI / UX community (12:23) Most knowledge work is not text generation (18:43) Finding the right constraints and interface for AI (21:27) Linus' journey to working at Notion (25:51) The importance of notations and interfaces (28:29) Setting interface defaults and standards (34:58) The challenges of designing AI agents (42:00) Notion deep dive: “Blocks”, AI, and more (53:22) Prompt engineering at Notion (01:04:22) Lightning Round SPONSOR: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co.

Nathan Labenz sits down with Lili Yu, a researcher of Meta AI to discuss the paper she authored: MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers. In this conversation, they discuss the architecture and breakthroughs of their research, and the opportunity to eliminate the need for tokenization. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics The Cognitive Revolution is a part of the Turpentine podcast network. Learn more: www.turpentine.co LINK: MEGABYTE Paper: https://arxiv.org/pdf/2305.07185.pdf TIMESTAMPS: (00:00) Episode preview (07:41) Takeaways from Lili Yu's paper: MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers (17:00) Architecture (24:59) Embeddings (27:43) Different local models (34:23) Encoder model (36:35) Transformer Architecture (48:10) Choosing patch size (01:08) What happens when you scale up? (01:19:20) Big picture for Meta AI (01:22:57) Responsible AI (01:27:02) China and AI TWITTER: @labenz (Nathan) @liliyu_lili (Lili) @eriktorenberg (Erik) @cogrev_podcast SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky (www.omneky.com) for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co. MUSIC CREDIT: MusicLM:

Ziming Liu is a Physics PhD student at MIT and IAIFI, advised by Prof. Max Tegmark. Ziming’s research is at the intersection of AI and physics. Today’s discussion goes in-depth on Liu’s paper “Seeing is Believing” where he presents Brain-Inspired Modular Training (BIMT), a method for making neural networks more modular and interpretable. The ability to see modules visually can complement current mechanistic interpretability strategies. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics The Cognitive Revolution is a part of the Turpentine podcast network. To learn more: Turpentine.co Watch this episode on YouTube to see research figures and graphics: https://www.youtube.com/watch?v=VKxwlxb2XWw Links: Seeing is Believing, authors: Ziming Liu, Eric Gan, and Max Tegmark https://arxiv.org/pdf/2305.08746.pdf Ziming Liu: https://kindxiaoming.github.io/ TIMESTAMPS: (00:00) Episode preview (01:16) Nathan introduces Ziming Liu and his research (04:50) What motivated Ziming's research into neural networks and modularity (08:06) Key Visuals of "Seeing is Believing" (08:44) Biology analogy & paper overview (15:07) Sponsor: Omneky (26:08) Research precedents (30:00) Locality vs modularity (37:40) Shoutout Tiny Stories : Episode 33 (39:00) Quantization Model of Neuro Scaling. (1:00:00) How can this work impact mechanistic interpretability strategies (1:27:00) Training (1:46:30) Downscaling large-scale models TWITTER: @labenz (Nathan) @ZimingLiu11 (Ziming) @cogrev_podcast SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky (www.omneky.com) for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co. MUSIC CREDIT: MusicLM

Nathan explores the Guaranteed Safe AI Framework with co-authors Ben Goldhaber and Nora Ammann. In this episode of The Cognitive Revolution, we discuss their groundbreaking position paper on ensuring robust and reliable AI systems. Join us for an in-depth conversation about the three-part system governing AI behavior and its potential impact on the future of AI safety. Apply to join over 400 founders and execs in the Turpentine Network: https://hmplogxqz0y.typeform.com/to/JCkphVqj RECOMMENDED PODCAST: 🎙️ Second Opinion - A new podcast for health-tech insiders from Christina Farr of the Second Opinion newsletter. Join Christina Farr, Luba Greenwood, and Ash Zenooz every week as they challenge industry experts with tough questions about the best bets in health-tech. Apple Podcasts: https://podcasts.apple.com/us/podcast/id1759267211 Spotify: https://open.spotify.com/show/0A8NwQE976s32zdBbZw6bv - 🎙️ History 102 with WhatifAltHist Every week, creator of WhatifAltHist Rudyard Lynch and Erik Torenberg cover a major topic in history in depth -- in under an hour. This season will cover classical Greece, early America, the Vikings, medieval Islam, ancient China, the fall of the Roman Empire, and more. Subscribe on Spotify: https://open.spotify.com/show/36Kqo3BMMUBGTDo1IEYihm Apple: https://podcasts.apple.com/us/podcast/history-102-with-whatifalthists-rudyard-lynch-and/id1730633913 YouTube: https://www.youtube.com/@History102-qg5oj SPONSORS: Oracle Cloud Infrastructure (OCI) is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds; offers one consistent price, and nobody does data better than Oracle. If you want to do more and spend less, take a free test drive of OCI at https://oracle.com/cognitive The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/ Head to Squad to access global engineering without the headache and at a fraction of the cost: head to https://choosesquad.com/ and mention “Turpentine” to skip the waitlist. CHAPTERS: (00:00:00) About the Show (00:04:39) Introduction (00:07:58) Convergence (00:10:32) Safety guarantees (00:14:35) World model (Part 1) (00:22:22) Sponsors: Oracle | Brave (00:24:31) World model (Part 2) (00:26:55) AI boxing (00:30:28) Verifier (00:33:33) Sponsors: Omneky | Squad (00:35:20) Example: Self-Driving Cars (00:38:08) Moral Desiderata (00:41:09) Trolley Problems (00:47:24) How to approach the world model (00:50:50) Deriving the world model (00:55:13) How far should the world model extend? (01:00:55) Safety through narrowness (01:02:38) Safety specs (01:08:26) Experiments (01:11:25) How GSAI can help in the short term (01:27:40) What would be the basis for the world model? (01:31:23) Interpretability (01:34:24) Competitive dynamics (01:37:35) Regulation (01:42:02) GSAI authors (01:43:25) Outro

Part 2 of Nathan's conversation with Tanishq Mathew Abraham focuses on Tanishq's work with virtual biopsy technology enabled by deep learning. This unique technology has the potential to address a number of important biomedical challenges; in particular, qOBM could help during live tumor and cancer removal surgeries. Tanishq, a 19-year-old UC Davis grad and one of the youngest people in the world to receive a Ph.D, with a degree in biomedical engineering, is the founder of the Medical AI Research Center (MedARC). In case you haven’t yet listened to part 1, check it out! That episode goes deep on Tanishq’s first published paper: Reconstructions of the Mind’s Eye, which encompasses breakthrough research on reconstructing visual perceptions from fMRI scans into images. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics The Cognitive Revolution is a part of the Turpentine podcast network. To learn more: Turpentine.co TIMESTAMPS: (00:00) Episode Preview (03:26) Introducing Tanishq’s second paper presenting an AI-enabled biopsy  (06:39) Diagnostic applications of this unique slide-free and label-free technology (qOBM) (13:06) Leveraging deep learning, specifically generative adversarial networks (GANs) (14:55) Sponsor: Omneky (15:42) Framework for model (16:22) Challenges of medical data sets (20:11) Challenges of unpaired image to image translation, addressed with a CycleGAN architecture  (22:19) What didn’t work (25:44) Breaking down GAN frameworks (34:30) Simplifying data to better with the CycleGAN (36:02) Factors for errors and confusing the model (39:00) Compute and training requirements (41:35) How this technology can scale in the near term and improve patient care (45:53) Tanishq’s relationships with EleutherAI and Stability AI (47:08) The future-looking focus of MedArc and unexplored opportunities (49:00) Developments in Medical AI field and interesting applications (55:11) Early education and AI (58:00) Would Tanishq get a neuralink? (1:00:00) Tanishq’s hopes and fears for AI LINKS: Paper: Label- and slide-free tissue histology using 3D epi-mode quantitative phase imaging and virtual H&E staining https://arxiv.org/pdf/2306.00548.pdf Authors: Tanishq Mathew Abraham, Paloma Casteleiro Costa, Caroline Filan, Zhe Guang, Zhaobin Zhang, Stewart Neill, Jeffrey J. Olson, Richard Levenson, Francisco E. Robles TWITTER: @iScienceLuvr (Tanishq) @MedARC_AI (MedARC) @CogRev_Podcast @labenz (Nathan) @eriktorenberg (Erik) SPONSOR: Thank you Omneky (www.omneky.com) for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. MUSIC CREDIT: MusicLM

In this episode, Nathan sits down with Tanishq Mathew Abraham, 19-year-old UC Davis grad and one of the youngest people in the world to receive a Ph.D, with a degree in biomedical engineering. Tanishq is the founder of the Medical AI Research Center (MedARC), and with his teammates, recently published a paper: Reconstructions of the Mind's Eye, which encompasses their breakthrough research on reconstructing visual perceptions from fMRI scans into images. In this episode, Nathan and Tanishq talk about the technology behind the fMRI-to-image project, developing the model, and future applications for this research. Part 2 with Tanishq will be released as the next episode. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics The Cognitive Revolution is a part of the Turpentine podcast network. To learn more: Turpentine.co TIMESTAMPS: (00:00) Episode Preview (05:43) The MindEye Project (09:06) Resemblance between AI reconstruction of mind's eye and visual presented (10:00) What is a voxel and which regions of the brain were studied? (10:23) What would the raw data of a voxel be? (11:44) Is there a time dimension to voxels? (15:00) Sponsor: Omneky (17:50) Goals for the MindEye project (25:57) What is the starting point of the model? (31:15) Aligning the model: reconstruction vs retrieval (40:34) Would doing a full end-to-end training be fine for the reconstruction? (42:15) The role of a limited data set (43:09) Training separate models per subject (45:07) Generalizability with a limited dataset (47:20) Mapping from one high-dimensional space to another (50:47) Stable Diffusion VAE encoding (1:00:50) How long does it take to train the model? (1:03:14) How similar or different are the subjects and their individual models? (1:05:59) The future of this research: custom models for your brain? (1:07:34) How much does this research contribute to brain research and wearables? (1:11:15) Fuzzing data and future research applications LINKS: MedARC: medarc.ai MindEye Paper: https://www.researchgate.net/publication/371136623_Reconstructing_the_Mind's_Eye_fMRI-to-Image_with_Contrastive_Learning_and_Diffusion_Priors MP3 of this episode: https://chrt.fm/track/993DGA/traffic.megaphone.fm/RINTP1584997572.mp3?updated=1687271014 TWITTER: @iScienceLuvr (Tanishq) @MedARC_AI (MedARC) @CogRev_Podcast @labenz (Nathan) @eriktorenberg (Erik) SPONSOR: Thank you Omneky (www.omneky.com) for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. MUSIC CREDIT: MusicLM

In this episode, Nathan sits down with Jonathan Frankle, Chief Scientist, and Abhi Venigalla, Research Scientist of MosaicML. They chat about Mosaic’s custom LLMs, the customers seeking Mosaic out and what their journeys and use cases look like, and exciting developments in Mosaic’s research: including their new inference platform, as well as Mosaic’s MPT-7B-65k+ storywriter model. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics The Cognitive Revolution is a part of the Turpentine podcast network. To learn more: www.turpentine.co TIMESTAMPS: (00:00) Episode Preview (06:04) Mosaic’s business model (07:28) Who uses Mosaic’s custom LLMs? What does their data look like? (09:55)Mosaic’s use cases for custom LLMs (12:47) How much extraction and summarization was done by humans pre-LLMs? (15:28) Sponsor: Omneky (21:50) The journeys of Mosaic’s customers and would a Wendy’s LLM know about a Big Mac? (25:46) The curriculum model and fine-tuning (29:10) Language models in the life sciences (33:20) How raw can data be before it becomes a problem? (35:44) Using the output of bulk pre-training process vs additional after training (38:30) Redteaming as a service (39:40) Mosaic’s inference platform (41:53) Spending one cent on 20,000 tokens, how is that cent distributed? (46:00)) Selling compute on a dedicated capacity basis (47:30) Oracle and AWS (49:50) The storywriter model and 65,000 token window (54:35) The transition from finite parameters into infinite attention matrix LINKS: MosaicML: https://www.mosaicml.com/ MPT-7B Storywriter Model: https://huggingface.co/mosaicml/mpt-7b-storywriter TWITTER: @jefrankle (Jonathan) @abhi_venigalla (Abhi) @MosaicML (Mosaic) @CogRev_Podcast @labenz (Nathan) @eriktorenberg (Erik) SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co. Music Credit: MusicLM

In this episode, Nathan Labenz and Erik Torenberg host a discussion between Emad Mostaque, founder of Stability AI, and Sam Lessin, GP of Slow Ventures and former VP Product at Facebook. These influential technology thinkers tackle topics pertinent to evaluating whether AI is a good investment for venture capital. They talk about the attention and information economy, SaaS markets, how AI-created entertainment will impact society and our relationships, and AI disruption of highly-regulated industries. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. LINK FOR NATHAN'S LIVE EVENT IN SAN FRANCISCO: Friday June 16th: Why AI with Athena: https://v4p9mjmurpv.typeform.com/to/llRcJSF9 TIMESTAMP: (00:00) Episode Preview (01:17) Nathan explains why we hosted a conversation with Emad and Sam (08:25) Discussion kicks off: AI as an extender rather than a deck shuffler (12:01) Is AI a disruptive or sustaining innovation? (14:22) What is intelligence? (16:00) The attention and information economy (21:56) Storytelling, the stock market, and the economy (23:23) Sponsor: Omneky (23:50) AI in the media industry, Hollywood, and distribution (29:10) Startups vs incumbents (30:30) Disrupting highly-regulated industries (32:30) AI in education: an AI tutor for every child (35:15) AI as StackOverflow 2.0 for engineers (37:23) Regulatory Arbitrage (40:06) The oracle problem in AI (42:00) Apple as the weakest BigTech player in AI (44:30) AI in communities and the future of social media (46:55) Replika on Valentine’s Day and AI-generated porn (53:50) How does the market for inference shape up? Inference vs. cloud (55:08) Price of primary care in US in the next decade (55:38) What are the most likely incumbents to be disrupted? (56:55) Will BigTech continue to opensource? (58:56) Marriage rate in US in the future given AI (01:00:24) Can any AI leader sustain high gross margins in the future? (01:02:04) Stability’s business model TWITTER: @EMostaque (Emad) @Lessin (Sam) @CogRev_Podcast @labenz (Nathan) @eriktorenberg (Erik) SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co. Download this episode as an mp3: https://chrt.fm/track/993DGA/traffic.megaphone.fm/RINTP2126380296.mp3?updated=1686676812 Music Credit: MusicLM

In this episode, Nathan sits down with Joshua Browder of DoNotPay, the world’s first robot lawyer. They chat about the current state of AI use in law, what policymakers should consider in regulating AI, and the ethics of robo-lawyers for consumer use.  We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics TIMESTAMPS: (00:00) Episode Preview (03:22) Joshua Browder and the story of DoNotPay (04:50) The value incurred through an AI lawyer (07:17) What are the legal and financial injustices that motivates Joshua’s work? (12:46) How does the AI negotiate with a Comcast chat agent? (16:28) Consumer security (17:12) Sponsor: Omneky (21:08) The ethics of DoNotPay (22:27) Should AI need to disclose that it’s AI? (24:47) How much human intervention is necessary in AI tooling? (27:15) Where does the burden of proof lie for AI regulation and ethics? (31:47) Advice to policymakers (35:45) AI Liability (38:05) AI Arms Race (43:46) Productive bot interactions (45:00) AI-powered arbitration (49:28) How the consumer experience might change two years from now (51:45) AI Superapps (55:25) Moats in AI (57:11) How much money does DoNotPay save people today? (59:30) Consumer and enterprise timelines given AI (01:06:30) What will AI replace? (01:09:34) Impact of AI in different industries (01:11:10) The legal field’s view of AI (01:15:50) Deflationary factors in running DoNotPay (01:25:00) What are non-obvious ways to save money? (01:27:55) Joshua’s favorite AI tools (01:28:17) Would Joshua get a Neuralink implant? LINKS: https://donotpay.com/ TWITTER: @jbrowder1 (Joshua) @DoNotPay (DoNotPay) @CogRev_Podcast @labenz (Nathan) @eriktorenberg (Erik) SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. NEWSLETTER: More show notes and reading material released in our Substack: https://cognitiverevolution.substack.com

Nathan Labenz sits down with Ronen Eldan and Yuanzhi Li of Microsoft Research to discuss the small natural language dataset they created called TinyStories. Tiny Stories is designed to reflect the full richness of natural language while still being small to support research with modest compute budgets.  Using this dataset, they began to explore aspects of language model performance, behavior, and mechanism by training a series of models that range in size from just 1 million to a maximum of 33 million parameters – which is still just 2% the scale of GPT-2. In this conversation, Nathan, Ronen, and Yuanzhi touch on LM reasoning, emergence, interpretability, and what understanding can be extended to LLMs. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics LINKS: Tiny Stories paper: https://huggingface.co/papers/2305.07759 TIMESTAMPS: (00:00) Episode Preview (07:12) The inspiration for the Tiny Stories project (15:07) Sponsor: Omneky (15:44) Creating the Tiny Stories dataset (21:27) GPT-4 vs GPT-3.5 (24:13) Did the TinyStories team try any other versions of GPT-4 (29:23) Curriculum models and weirder curriculums (35:34) What does reasoning mean? (46:27) What does emergence mean? (01:01:44) The curriculum development space (01:11:40) The similarities between models and human development (01:20:12) Fewer layers vs. more layers (01:29:22) Attention heads (01:33:40) Semantic attention head (01:36:54) Neuron technique used in developing the TinyStories model (01:52:20) Interpretability work that inspires Ronen and Yuanzhi TWITTER: @CogRev_Podcast @EldanRonen (Ronen) @labenz (Nathan) @eriktorenberg (Erik) SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co. Music Credit: MusicLM More show notes and reading material released in our Substack: https://cognitiverevolution.substack.com

This 20-min episode comes from our friend Nathaniel Whittemore's excellent daily podcast The AI Breakdown Podcast. This episode aired on June 1, 2023, and covers the latest developments from OpenAI, including new features, a cybersecurity grant program, and their new process rewards model for trading. We hope you enjoy it as much as we do. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics SPONSOR: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co. Subscribe to The AI Breakdown podcast: https://pod.link/1680633614 Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown

Nathan and Erik analyze the moats of the most powerful companies in AI. The paradigm-shifting technology has led to a flourishing open-source community with market share. Yet, the big players have key competitive advantages that can be examined from many different angles. LINKS: Nathan’s Twitter thread on AI moats that sparked this discussion: https://twitter.com/labenz/status/1654853321876815872 Read about the 9 moats at length in our newsletter: https://cognitiverevolution.substack.com/p/the-leaked-google-memo-and-a Leaderboard for AI rankings: https://lmsys.org/ -- We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics TIMESTAMPS: (00:00) Episode Preview (01:36) Where are the moats in AI? (07:31) Open source vs. closed (15:24) Recommendation: The AI Breakdown Podcast (16:39) Sponsor: Omneky (20:46) AI Safety (33:00) Which players are going to win? 9 Moats. TWITTER: @CogRev_Podcast @labenz (Nathan) @eriktorenberg (Erik) SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co. Music Credit: MusicLM More show notes and reading material released in our Substack: https://cognitiverevolution.substack.com

Today’s episode is a deep dive into the collision of AI and the future of entertainment, against the backdrop of the still-raging Writers Guild of America (WGA) strike. Nathan Labenz sits down with Trey Kollmer, Sophia Lear, and Garrett Schabb – all seasoned television writers and Guild members – to discuss the labor dispute. While the strike encompasses many dynamics, the timely intersection with a rapidly changing AI landscape has the writers entwined in the wild possibilities and existential threats of paradigm-shifting technology. Other entertainment guilds and labor unions are watching closely. Note: Nathan kicks off this episode with a longer-than-usual introduction and analysis as the subject matter of this episode requires some added context. If you would prefer to dive straight into the interviews, follow the timecodes below to skip ahead (and maybe give the intro a listen when you finish!) More about our guests Trey Kollmer, Sophia Lear, and Garrett Schabb and their work: Trey Kollmer, currently co-executive producer for the TV show Ghosts, was described to me as perhaps the single most knowledgeable guild member on the topic of AI. Follow him at @treyko on Twitter. Sophia Lear has been a writer for TV shows including Ghosts, The Unicorn, and New Girl, and was also previously an assistant literary editor for the New Republic. Garrett Schabb has written for shows including Tosh.0 and Suits (of Meghan Markle fame), and has also written for Crooked Media. Follow him at @garrettschabb TIMESTAMPS: (00:00) Preview (01:54) Introduction and analysis for this episode  (09:02) Trey Kollmer breaks down the dynamics of compensation, ownership and crediting for WGA writers, issues at the heart of the strike (14:52) Sponsor: Omneky (15:09) Recommendation: The AI Breakdown (22:42) Usefulness of AI for script writing (and will better training out of jailbreaks make it less useful?)  (30:59) The main two AI-related demands of the labor dispute (42:48) Writers’ diverse reactions to AI, from hostility to experimentation (50:00) Picturing a long term Utopian future (57:46) Sophia Lear breaks down sentiments around GPT-4, AI-generated scripts and the previous eras of crappy sitcoms and bad streaming series (written by humans) (1:11:27) Whose jobs are safe? (1:17:03) Garrett Schabb breaks down dystopian/realistic view of studio’s incentives (1:20:33) How Garrett uses ChatGPT (1:27:37) Ethical boundaries of using the model to write like your favorite writers (1;32:16) The two types of storytelling in the future, UBI and decoupling writing from a profession SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. Music Credit: MusicLM More show notes and reading material released in our Substack: https://cognitiverevolution.substack.com

Nathan Labenz sits down with Andrew Feldman, CEO and Co-Founder of Cerebras Systems, a company building a new class of computer system for accelerating AI and changing the future of work. Cerebras Systems is the creator of the world’s largest chip, at 2.6 trillion transistors. In this episode, they discuss the founding story of Cerebras, the experience of creating the world’s largest chip, and the process that goes into chip design and manufacturing for an AI-focused chip.  This episode is the first part of our hardware exploration series focused on the people building at the forefront of hardware applications in AI. LINKS: Cerebras: https://www.cerebras.net/ Book: The Chip War by Chris Miller We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. PODCAST RECOMMENDATION: The AI Breakdown: https://pod.link/1680633614 As anyone in AI knows, the pace of progress of new releases is relentless. The AI Breakdown is a daily podcast (10-20min long) that helps us ensure we don't miss anything important by curating news and analysis. TIMESTAMPS (00:00) Preview (04:27) Andrew’s story of creating the world’s largest chip and Cerebras (07:19) What is a chip? (08:14) The diversity of chips and what they can accomplish (09:47) What is it like to design a 2.5 trillion transistor chip? (12:41) The founding story of Cerebras and building the team (14:20) Sponsor: Omneky (23:00) What was the hardest part about building the company? (26:11) What happens after designing the chip’s blueprint? (27:29) The tradeoffs needed in chipmaking  (34:08) The comparison between chips and neural networks (38:31) The generalization vs specialization of a chip  (40:11) Sparse compute vs dense compute (43:55) Ghost in the machine (46:54) Supply chain challenges of the Cerebras chip (54:59) The future for chips (58:19) Building chip clusters (58:57) The Cerebras business model (01:00:41) Building a chip cluster vs using a Cerebras chip (01:02:57) Giant chips on the edge (01:05:32) What is the edge?  (01:08:04) Andrew’s favorite AI products (01:10:08) Would Andrew get a Neuralink implant? (01:14:16) Consciousness and chips (01:17:50) AI hopes and fears TWITTER: @CogRev_Podcast @andrewdfeldman (Andrew) @labenz (Nathan) @eriktorenberg (Erik) SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co. Music Credit: MusicLM More show notes and reading material released in our Substack: https://cognitiverevolution.substack.com

Nathan interviews Mike Knoop, co-founder of Zapier and co-creator of the ARC Prize, about the $1 million competition for more efficient AI architectures. They discuss the ARC AGI benchmark, its implications for general intelligence, and the potential impact on AI safety. Nathan reflects on the challenges of intuitive problem-solving in AI and considers hybrid approaches to AGI development. Apply to join over 400 founders and execs in the Turpentine Network: https://hmplogxqz0y.typeform.com/to/JCkphVqj RECOMMENDED PODCAST 🎙️ Second Opinion - A new podcast for health-tech insiders from Christina Farr of the Second Opinion newsletter. Join Christina Farr, Luba Greenwood, and Ash Zenooz every week as they challenge industry experts with tough questions about the best bets in health-tech. Apple Podcasts: https://podcasts.apple.com/us/podcast/id1759267211 Spotify: https://open.spotify.com/show/0A8NwQE976s32zdBbZw6bv - 🎙️ History 102 with WhatifAltHist Every week, creator of WhatifAltHist Rudyard Lynch and Erik Torenberg cover a major topic in history in depth -- in under an hour. This season will cover classical Greece, early America, the Vikings, medieval Islam, ancient China, the fall of the Roman Empire, and more. Subscribe on Spotify: https://open.spotify.com/show/36Kqo3BMMUBGTDo1IEYihm Apple: https://podcasts.apple.com/us/podcast/history-102-with-whatifalthists-rudyard-lynch-and/id1730633913 YouTube: https://www.youtube.com/@History102-qg5oj SPONSORS: Oracle Cloud Infrastructure (OCI) is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds; offers one consistent price, and nobody does data better than Oracle. If you want to do more and spend less, take a free test drive of OCI at https://oracle.com/cognitive The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/ Head to Squad to access global engineering without the headache and at a fraction of the cost: head to https://choosesquad.com/ and mention “Turpentine” to skip the waitlist. CHAPTERS: (00:00:00) About the Show (00:06:06) The ARC Benchmark (00:09:34) Other Benchmarks (00:10:58) Definition of AGI (00:14:38) The rules of the contest (00:18:16) ARC test set (Part 1) (00:18:23) Sponsors: Oracle | Brave (00:20:31) ARC test set (Part 2) (00:22:50) Stair-stepping benchmarks (00:26:17) ARC Prize (00:28:34) The rules of the ARC Prize (00:31:12) Compute costs (Part 1) (00:34:47) Sponsors: Omneky | Squad (00:36:34) Compute costs (Part 2) (00:51:20) Intuition (00:54:32) Human Intelligence (00:56:06) Current Frontier Language Models (00:57:44) Program Synthesis (01:04:10) Is the model learning or memorizing? (01:15:02) Exploring Solutions (01:17:02) Non-backpropagation evolutionary architecture search (01:19:49) Expectations for an AGI world (01:24:11) Reliability and out of domain generalization (01:28:35) What a person would do (01:29:51) What is the right generalization (01:35:32) The ARC AGI Challenge (01:48:32) FunSearch (01:50:41) Kolmogorov-Arnold-Networks (01:54:18) Grokking (01:55:42) Outro

Nathan Labenz sits down with engineer Shreya Rajpal, the creator of Guardrails AI, a new Python library that allows developers to add a layer of output, validation and correction to their code. Practically guardrails can ensure a reliable interface between language models and more traditional deterministic software systems. At the same time, mind-bending and potentially risky use case frameworks like Guardrails allow developers to ask and answer entirely new kinds of questions. Talking to Shreya really reinforced just how early we are in LLM’s impact on the software industry. We were introduced to Shreya’s work when recording our interview with Matt Welsh, the CEO of Fixie.AI (Ep 19: How AI Agents Will Change How We Work). LINKS: Guardrails AI: https://shreyar.github.io/guardrails/ We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. TIMESTAMPS: (00:00) Episode preview (05:00) Why Shreya built Guardrails AI (08:33) Common ways LMs can “go off the rails” and how Guardrails can correct it (13:45) Discussion of validators (15:31) Sponsor: Omneky (18:48) Business and creative use cases of Guardrails AI (25:00) What can be achieved by Guardrails AI that cannot be achieved by traditional code (32:44) How agents work vs how Guardrails works (35:34) AI as shepherd vs delegating to AI and the role of human understanding (39:54) Trust deficit and risks (46:33) Is it realistic to imagine GPT-4 using Guardrails? (52:00) How Shreya thinks about security (57:02) How Shreya thinks about embeddings (1:05:05) How Shreya thinks about problemsolving with LMs (1:07:50) Shreya on OpenAI’s Evals Library, Anthropic’s Constitutional AI (1:12:55) Discussion of determinism (1:15:00) Recommendations for developers to minimize overhead (1:26:00) Predictions about providers in the space of LMs (1:29:30) Shreya’s favorite AI tools (1:30:00) Would Shreya get a neuralink implant (1:31:50) Biggest hopes and fears for AI TWITTER: @CogRev_Podcast @ShreyaR (Shreya) @labenz (Nathan) @eriktorenberg (Erik) SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co. Music Credit: MusicLM More show notes and reading material released in our Substack: https://cognitiverevolution.substack.com

Nathan sits down with Vivek Natarajan, research scientist at Google Health. Vivek leads the Google Brain moonshot behind Med-PaLM, Google’s flagship medical large language model, featured in The Economist, The Scientific American, CNBC, and Forbes. In this episode, they discuss the foundational models that Vivek and team built before Med-PaLM, the techniques used to develop Med-PaLM which will be of interest to anyone developing AI systems for high-stakes use cases, and the capabilities for Med-PaLM to equalize access to medical knowledge and care. This episode is part of a series centered on talking to the people at the cutting edge of building AI-driven solutions in medicine. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. LINKS: https://sites.research.google/med-palm/ FEEDBACK / COLLABORATE WITH NATHAN: Email: TCR@turpentine.co TIMESTAMPS: (00:00) Episode preview (03:43) The story of how Med-PaLM came to be (09:41) Building Med-PaLM’s infrastructure (13:10) The US medical licensing exam as a measure of AI progress (15:23) Sponsor: Omneky (18:17) Practicality of benchmarking in real-world usage (21:39) Overcoming the shortfalls of Flan-PaLM with Med-PaLM (25:08) Choosing to use soft prompting over few shot prompting (30:36) The process of training Flan-PaLM (37:31) A curriculum approach to soft-prompting (38:43) Layperson vs expert interactions with LLMs (43:54) How did the Google team facilitate user exploration of the model’s capabilities? (46:58) Shift in techniques from Med-PaLM to Med-PaLM2 (50:21) Using different prompting strategies with Med-PaLM2 (57:33) Is Med-PaLM 2 preferred over clinicians? (01:02:28) Will there be a multimodal version of Med-PaLM? (01:04:52) Breakthroughs required for AI to further advance human potential (01:10:23) The Med-PaLM business plan (01:12:08) Is there a vision for a consumer product? (01:15:46) The pros and cons of pre-training a model (01:19:45) Vivek’s favorite AI products (01:21:01) Would Vivek get a Neuralink implant? (01:23:08) AI hopes and fears TWITTER: @CogRev_Podcast @vivnat (Vivek) @labenz (Nathan) @eriktorenberg (Erik) SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co. Music Credit: MusicLM More show notes and reading material released in our Substack: https://cognitiverevolution.substack.com/

[Bonus Episode] Future of Life Institute Podcast host Gus Docker interviews Conjecture CEO Connor Leahy to discuss GPT-4, magic, cognitive emulation, demand for human-like AI, and aligning superintelligence. You can read more about Connor's work at https://conjecture.dev Future of Life Institute is the organization that recently published an open letter calling for a six-month pause on training new AI systems. FLI was founded by Jann Tallinn who we interviewed in Episode 16 of The Cognitive Revolution. We think their podcast is excellent. They frequently interview critical thinkers in AI like Neel Nanda, Ajeya Cotra, and Connor Leahy - an episode we found particularly fascinating and is airing for our audience today. The FLI Podcast also recently interviewed Nathan Labenz for a 2-part episode: https://futureoflife.org/podcast/nathan-labenz-on-how-ai-will-transform-the-economy/ SUBSCRIBE: Future of Life Institute Podcast: Apple: https://podcasts.apple.com/us/podcast/future-of-life-institute-podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP -- We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord. https://link.chtbl.com/hrheretics TIMESTAMPS: (00:00) Episode introduction (01:55) GPT-4  (18:30) "Magic" in machine learning  (29:43) Cognitive emulations  (40:00) Machine learning VS explainability  (49:50) Human data = human AI?  (1:01:50) Analogies for cognitive emulations  (1:28:10) Demand for human-like AI  (1:33:50) Aligning superintelligence  SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co. If you'd like to listen to Part 2 of this interview with Connor Leahy, you can head here:  https://podcasts.apple.com/us/podcast/connor-leahy-on-the-state-of-ai-and-alignment-research/id1170991978?i=1000609972001