Podcast AI Engineering Podcast par Tobias Macey Épisodes

Explorez tous les épisodes du podcast AI Engineering Podcast

Plongez dans la liste complète des épisodes de AI Engineering Podcast. Chaque épisode est catalogué accompagné de descriptions détaillées, ce qui facilite la recherche et l'exploration de sujets spécifiques. Suivez tous les épisodes de votre podcast préféré et ne manquez aucun contenu pertinent.

	Titre	Date	Durée
	The Role Of Synthetic Data In Building Better AI Applications	16 Feb 2025	00:54:21
Summary In this episode of the AI Engineering Podcast Ali Golshan, co-founder and CEO of Gretel.ai, talks about the transformative role of synthetic data in AI systems. Ali explains how synthetic data can be purpose-built for AI use cases, emphasizing privacy, quality, and structural stability. He highlights the shift from traditional methods to using language models, which offer enhanced capabilities in understanding data's deep structure and generating high-quality datasets. The conversation explores the challenges and techniques of integrating synthetic data into AI systems, particularly in production environments, and concludes with insights into the future of synthetic data, including its application in various industries, the importance of privacy regulations, and the ongoing evolution of AI systems. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Seamless data integration into AI applications often falls short, leading many to adopt RAG methods, which come with high costs, complexity, and limited scalability. Cognee offers a better solution with its open-source semantic memory engine that automates data ingestion and storage, creating dynamic knowledge graphs from your data. Cognee enables AI agents to understand the meaning of your data, resulting in accurate responses at a lower cost. Take full control of your data in LLM apps without unnecessary overhead. Visit aiengineeringpodcast.com/cognee to learn more and elevate your AI apps and agents. Your host is Tobias Macey and today I'm interviewing Ali Golshan about the role of synthetic data in building, scaling, and improving AI systems Interview Introduction How did you get involved in machine learning? Can you start by summarizing what you mean by synthetic data in the context of this conversation? How have the capabilities around the generation and integration of synthetic data changed across the pre- and post-LLM timelines? What are the motivating factors that would lead a team or organization to invest in synthetic data generation capacity? What are the main methods used for generation of synthetic data sets? How does that differ across open-source and commercial offerings? From a surface level it seems like synthetic data generation is a straight-forward exercise that can be owned by an engineering team. What are the main "gotchas" that crop up as you move along the adoption curve? What are the scaling characteristics of synthetic data generation as you go from prototype to production scale? domains/data types that are inappropriate for synthetic use cases (e.g. scientific or educational content) managing appropriate distribution of values in the generation process Beyond just producing large volumes of semi-random data (structured or otherwise), what are the other processes involved in the workflow of synthetic data and its integration into the different systems that consume it? What are the most interesting, innovative, or unexpected ways that you have seen synthetic data generation used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on synthetic data generation? When is synthetic data the wrong choice? What do you have planned for the future of synthetic data capabilities at Gretel? Contact Info LinkedIn Parting Question From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links Gretel Hadoop LSTM == Long Short-Term Memory GAN == Generative Adversarial Network Textbooks are all you need MSFT paper Illumina The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Optimize Your AI Applications Automatically With The TensorZero LLM Gateway	22 Jan 2025	01:03:05
Summary In this episode of the AI Engineering podcast Viraj Mehta, CTO and co-founder of TensorZero, talks about the use of LLM gateways for managing interactions between client-side applications and various AI models. He highlights the benefits of using such a gateway, including standardized communication, credential management, and potential features like request-response caching and audit logging. The conversation also explores TensorZero's architecture and functionality in optimizing AI applications by managing structured data inputs and outputs, as well as the challenges and opportunities in automating prompt generation and maintaining interaction history for optimization purposes. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Seamless data integration into AI applications often falls short, leading many to adopt RAG methods, which come with high costs, complexity, and limited scalability. Cognee offers a better solution with its open-source semantic memory engine that automates data ingestion and storage, creating dynamic knowledge graphs from your data. Cognee enables AI agents to understand the meaning of your data, resulting in accurate responses at a lower cost. Take full control of your data in LLM apps without unnecessary overhead. Visit aiengineeringpodcast.com/cognee to learn more and elevate your AI apps and agents. Your host is Tobias Macey and today I'm interviewing Viraj Mehta about the purpose of an LLM gateway and his work on TensorZero Interview Introduction How did you get involved in machine learning? What is an LLM gateway? What purpose does it serve in an AI application architecture? What are some of the different features and capabilities that an LLM gateway might be expected to provide? Can you describe what TensorZero is and the story behind it? What are the core problems that you are trying to address with Tensor0 and for whom? One of the core features that you are offering is management of interaction history. How does this compare to the "memory" functionality offered by e.g. LangChain, Cognee, Mem0, etc.? How does the presence of TensorZero in an application architecture change the ways that an AI engineer might approach the logic and control flows in a chat-based or agent-oriented project? Can you describe the workflow of building with Tensor0 and some specific examples of how it feeds back into the performance/behavior of an LLM? What are some of the ways in which the addition of Tensor0 or another LLM gateway might have a negative effect on the design or operation of an AI application? What are the most interesting, innovative, or unexpected ways that you have seen TensorZero used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on TensorZero? When is TensorZero the wrong choice? What do you have planned for the future of TensorZero? Contact Info LinkedIn Parting Question From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links TensorZero LLM Gateway LiteLLM OpenAI Google Vertex Anthropic Reinforcement Learning Tokamak Reactor Viraj RLHF Paper Contextual Dueling Bandits Direct Preference Optimization Partially Observable Markov Decision Process DSPy PyTorch Cognee Mem0 LangGraph Douglas Hofstadter OpenAI Gym OpenAI o1 OpenAI o3 Chain Of Thought The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Harnessing Generative AI for Effective Digital Advertising Campaigns	02 Sep 2024	00:41:49
Summary In this episode of the AI Engineering podcast Praveen Gujar, Director of Product at LinkedIn, talks about the applications of generative AI in digital advertising. He highlights the key areas of digital advertising, including audience targeting, content creation, and ROI measurement, and delves into how generative AI is revolutionizing these aspects. Praveen shares successful case studies of generative AI in digital advertising, including campaigns by Heinz, the Barbie movie, and Maggi, and discusses the potential pitfalls and risks associated with AI-powered tools. He concludes with insights into the future of generative AI in digital advertising, highlighting the importance of cultural transformation and the synergy between human creativity and AI. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Your host is Tobias Macey and today I'm interviewing Praveen Gujar about the applications of generative AI in digital advertising Interview Introduction How did you get involved in machine learning? Can you start by defining "digital advertising" for the scope of this conversation? What are the key elements/characteristics/goals of digital avertising? In the world before generative AI, what did a typical end-to-end advertising campaign workflow look like? What are the stages of that workflow where generative AI are proving to be most useful? How do the current limitations of generative AI (e.g. hallucinations, non-determinism) impact the ways in which they can be used? What are the technological and organizational systems that need to be implemented to effectively apply generative AI in public-facing applications that are so closely tied to brand/company image? What are the elements of user education/expectation setting that are necessary when working with marketing/advertising personnel to help avoid damage to the brands? What are some examples of applications for generative AI in digital advertising that have gone well? Any that have gone wrong? What are the most interesting, innovative, or unexpected ways that you have seen generative AI used in digital advertising? What are the most interesting, unexpected, or challenging lessons that you have learned while working on digital advertising applications of generative AI? When is generative AI the wrong choice? What are your future predictions for the use of generative AI in dgital advertising? Contact Info Website LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links Generative AI LLM == Large Language Model Dall-E) RLHF == Reinforcement Learning fHuman Feedback The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Building Scalable ML Systems on Kubernetes	15 Aug 2024	00:50:22
Summary In this episode of the AI Engineering podcast, host Tobias Macy interviews Tammer Saleh, founder of SuperOrbital, about the potentials and pitfalls of using Kubernetes for machine learning workloads. The conversation delves into the specific needs of machine learning workflows, such as model tracking, versioning, and the use of Jupyter Notebooks, and how Kubernetes can support these tasks. Tammer emphasizes the importance of a unified API for different teams and the flexibility Kubernetes provides in handling various workloads. Finally, Tammer offers advice for teams considering Kubernetes for their machine learning workloads and discusses the future of Kubernetes in the ML ecosystem, including areas for improvement and innovation. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Your host is Tobias Macey and today I'm interviewing Tammer Saleh about the potentials and pitfalls of using Kubernetes for your ML workloads. Interview Introduction How did you get involved in Kubernetes? For someone who is unfamiliar with Kubernetes, how would you summarize it? For the context of this conversation, can you describe the different phases of ML that we're talking about? Kubernetes was originally designed to handle scaling and distribution of stateless processes. ML is an inherently stateful problem domain. What challenges does that add for K8s environments? What are the elements of an ML workflow that lend themselves well to a Kubernetes environment? How much Kubernetes knowledge does an ML/data engineer need to know to get their work done? What are the sharp edges of Kubernetes in the context of ML projects? What are the most interesting, unexpected, or challenging lessons that you have learned while working with Kubernetes? When is Kubernetes the wrong choice for ML? What are the aspects of Kubernetes (core or the ecosystem) that you are keeping an eye on which will help improve its utility for ML workloads? Contact Info Email LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for ML workloads today? Links SuperOrbital CloudFoundry Heroku 12 Factor Model Kubernetes Docker Compose Core K8s Class Jupyter Notebook Crossplane Ochre Jelly CNCF (Cloud Native Computing Foundation) Landscape Stateful Set RAG == Retrieval Augmented Generation Podcast Episode Kubeflow Flyte Data Engineering Podcast Episode Pachyderm Data Engineering Podcast Episode CoreWeave Kubectl ("koob-cuddle") Helm CRD == Custom Resource Definition Horovod Podcast.__init__ Episode Temporal Slurm Ray Dask Infiniband The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA
	Expert Insights On Retrieval Augmented Generation And How To Build It	28 Jul 2024	01:03:21
Summary In this episode we're joined by Matt Zeiler, founder and CEO of Clarifai, as he dives into the technical aspects of retrieval augmented generation (RAG). From his journey into AI at the University of Toronto to founding one of the first deep learning AI companies, Matt shares his insights on the evolution of neural networks and generative models over the last 15 years. He explains how RAG addresses issues with large language models, including data staleness and hallucinations, by providing dynamic access to information through vector databases and embedding models. Throughout the conversation, Matt and host Tobias Macy discuss everything from architectural requirements to operational considerations, as well as the practical applications of RAG in industries like intelligence, healthcare, and finance. Tune in for a comprehensive look at RAG and its future trends in AI. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Your host is Tobias Macey and today I'm interviewing Matt Zeiler, Founder & CEO of Clarifai, about the technical aspects of RAG, including the architectural requirements, edge cases, and evolutionary characteristics Interview Introduction How did you get involved in the area of data management? Can you describe what RAG (Retrieval Augmented Generation) is? What are the contexts in which you would want to use RAG? What are the alternatives to RAG? What are the architectural/technical components that are required for production grade RAG? Getting a quick proof-of-concept working for RAG is fairly straightforward. What are the failures modes/edge cases that start to surface as you scale the usage and complexity? The first step of building the corpus for RAG is to generate the embeddings. Can you talk through the planning and design process? (e.g. model selection for embeddings, storage capacity/latency, etc.) How does the modality of the input/output affect this and downstream decisions? (e.g. text vs. image vs. audio, etc.) What are the features of a vector store that are most critical for RAG? The set of available generative models is expanding and changing at breakneck speed. What are the foundational aspects that you look for in selecting which model(s) to use for the output? Vector databases have been gaining ground for search functionality, even without generative AI. What are some of the other ways that elements of RAG can be re-purposed? What are the most interesting, innovative, or unexpected ways that you have seen RAG used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on RAG? When is RAG the wrong choice? What are the main trends that you are following for RAG and its component elements going forward? Contact Info Website LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. [Podcast.__init__]() covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links Clarifai Geoff Hinton Yann Lecun Neural Networks Deep Learning Retrieval Augmented Generation Context Window Vector Database Prompt Engineering Mistral Llama 3 Embedding Quantization Active Learning Google Gemini AI Model Attention Recurrent Network Convolutional Network Reranking Model Stop Words Massive Text Embedding Benchmark (MTEB) Retool State of AI Report pgvector Milvus Qdrant Pinecone OpenLLM Leaderboard Semantic Search Hashicorp The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Barking Up The Wrong GPTree: Building Better AI With A Cognitive Approach	28 Jul 2024	00:52:49
Summary Artificial intelligence has dominated the headlines for several months due to the successes of large language models. This has prompted numerous debates about the possibility of, and timeline for, artificial general intelligence (AGI). Peter Voss has dedicated decades of his life to the pursuit of truly intelligent software through the approach of cognitive AI. In this episode he explains his approach to building AI in a more human-like fashion and the emphasis on learning rather than statistical prediction. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Your host is Tobias Macey and today I'm interviewing Peter Voss about what is involved in making your AI applications more "human" Interview Introduction How did you get involved in machine learning? Can you start by unpacking the idea of "human-like" AI? How does that contrast with the conception of "AGI"? The applications and limitations of GPT/LLM models have been dominating the popular conversation around AI. How do you see that impacting the overrall ecosystem of ML/AI applications and investment? The fundamental/foundational challenge of every AI use case is sourcing appropriate data. What are the strategies that you have found useful to acquire, evaluate, and prepare data at an appropriate scale to build high quality models? What are the opportunities and limitations of causal modeling techniques for generalized AI models? As AI systems gain more sophistication there is a challenge with establishing and maintaining trust. What are the risks involved in deploying more human-level AI systems and monitoring their reliability? What are the practical/architectural methods necessary to build more cognitive AI systems? How would you characterize the ecosystem of tools/frameworks available for creating, evolving, and maintaining these applications? What are the most interesting, innovative, or unexpected ways that you have seen cognitive AI applied? What are the most interesting, unexpected, or challenging lessons that you have learned while working on desiging/developing cognitive AI systems? When is cognitive AI the wrong choice? What do you have planned for the future of cognitive AI applications at Aigo? Contact Info LinkedIn Website Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links Aigo.ai Artificial General Intelligence Cognitive AI Knowledge Graph Causal Modeling Bayesian Statistics Thinking Fast & Slow by Daniel Kahneman (affiliate link) Agent-Based Modeling Reinforcement Learning DARPA 3 Waves of AI presentation Why Don't We Have AGI Yet? whitepaper Concepts Is All You Need Whitepaper Hellen Keller Stephen Hawking The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Build Your Second Brain One Piece At A Time	28 Jul 2024	00:48:27
Summary Generative AI promises to accelerate the productivity of human collaborators. Currently the primary way of working with these tools is through a conversational prompt, which is often cumbersome and unwieldy. In order to simplify the integration of AI capabilities into developer workflows Tsavo Knott helped create Pieces, a powerful collection of tools that complements the tools that developers already use. In this episode he explains the data collection and preparation process, the collection of model types and sizes that work together to power the experience, and how to incorporate it into your workflow to act as a second brain. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Your host is Tobias Macey and today I'm interviewing Tsavo Knott about Pieces, a personal AI toolkit to improve the efficiency of developers Interview Introduction How did you get involved in machine learning? Can you describe what Pieces is and the story behind it? The past few months have seen an endless series of personalized AI tools launched. What are the features and focus of Pieces that might encourage someone to use it over the alternatives? model selections architecture of Pieces application local vs. hybrid vs. online models model update/delivery process data preparation/serving for models in context of Pieces app application of AI to developer workflows types of workflows that people are building with pieces What are the most interesting, innovative, or unexpected ways that you have seen Pieces used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Pieces? When is Pieces the wrong choice? What do you have planned for the future of Pieces? Contact Info LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links Pieces NPU == Neural Processing Unit Tensor Chip LoRA == Low Rank Adaptation Generative Adversarial Networks Mistral Emacs Vim NeoVim Dart Flutter Typescript Lua Retrieval Augmented Generation ONNX LSTM == Long Short-Term Memory LLama 2 GitHub Copilot Tabnine Podcast Episode The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Strategies For Building A Product Using LLMs At DataChat	03 Mar 2024	00:48:41
Summary Large Language Models (LLMs) have rapidly captured the attention of the world with their impressive capabilities. Unfortunately, they are often unpredictable and unreliable. This makes building a product based on their capabilities a unique challenge. Jignesh Patel is building DataChat to bring the capabilities of LLMs to organizational analytics, allowing anyone to have conversations with their business data. In this episode he shares the methods that he is using to build a product on top of this constantly shifting set of technologies. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Jignesh Patel about working with LLMs; understanding how they work and how to build your own Interview Introduction How did you get involved in machine learning? Can you start by sharing some of the ways that you are working with LLMs currently? What are the business challenges involved in building a product on top of an LLM model that you don't own or control? In the current age of business, your data is often your strategic advantage. How do you avoid losing control of, or leaking that data while interfacing with a hosted LLM API? What are the technical difficulties related to using an LLM as a core element of a product when they are largely a black box? What are some strategies for gaining visibility into the inner workings or decision making rules for these models? What are the factors, whether technical or organizational, that might motivate you to build your own LLM for a business or product? Can you unpack what it means to "build your own" when it comes to an LLM? In your work at DataChat, how has the progression of sophistication in LLM technology impacted your own product strategy? What are the most interesting, innovative, or unexpected ways that you have seen LLMs/DataChat used? What are the most interesting, unexpected, or challenging lessons that you have learned while working with LLMs? When is an LLM the wrong choice? What do you have planned for the future of DataChat? Contact Info Website LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links DataChat CMU == Carnegie Mellon University SVM == Support Vector Machine Generative AI Genomics Proteomics Parquet OpenAI Codex LLama Mistral Google Vertex Langchain Retrieval Augmented Generation Prompt Engineering Ensemble Learning XGBoost Catboost Linear Regression COGS == Cost Of Goods Sold Bruce Schneier - AI And Trust The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Improve The Success Rate Of Your Machine Learning Projects With bizML	18 Feb 2024	00:50:22
Summary Machine learning is a powerful set of technologies, holding the potential to dramatically transform businesses across industries. Unfortunately, the implementation of ML projects often fail to achieve their intended goals. This failure is due to a lack of collaboration and investment across technological and organizational boundaries. To help improve the success rate of machine learning projects Eric Siegel developed the six step bizML framework, outlining the process to ensure that everyone understands the whole process of ML deployment. In this episode he shares the principles and promise of that framework and his motivation for encapsulating it in his book "The AI Playbook". Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Eric Siegel about how the bizML approach can help improve the success rate of your ML projects Interview Introduction How did you get involved in machine learning? Can you describe what bizML is and the story behind it? What are the key aspects of this approach that are different from the "industry standard" lifecycle of an ML project? What are the elements of your personal experience as an ML consultant that helped you develop the tenets of bizML? Who are the personas that need to be involved in an ML project to increase the likelihood of success? Who do you find to be best suited to "own" or "lead" the process? What are the organizational patterns that might hinder the work of delivering on the goals of an ML initiative? What are some of the misconceptions about the work involved in/capabilities of an ML model that you commonly encounter? What is your main goal in writing your book "The AI Playbook"? What are the most interesting, innovative, or unexpected ways that you have seen the bizML process in action? What are the most interesting, unexpected, or challenging lessons that you have learned while working on ML projects and developing the bizML framework? When is bizML the wrong choice? What are the future developments in organizational and technical approaches to ML that will improve the success rate of AI projects? Contact Info LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links The AI Playbook: Mastering the Rare Art of Machine Learning Deployment by Eric Siegel Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die by Eric Siegel Columbia University Machine Learning Week Conference Generative AI World Machine Learning Leadership and Practice Course Rexer Analytics KD Nuggets CRISP-DM Random Forest Gradient Descent The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Using Generative AI To Accelerate Feature Engineering At FeatureByte	11 Feb 2024	00:44:59
Summary One of the most time consuming aspects of building a machine learning model is feature engineering. Generative AI offers the possibility of accelerating the discovery and creation of feature pipelines. In this episode Colin Priest explains how FeatureByte is applying generative AI models to the challenge of building and maintaining machine learning pipelines. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Colin Priest about applying generative AI to the task of building and deploying AI pipelines Interview Introduction How did you get involved in machine learning? Can you start by giving the 30,000 foot view of the steps involved in an AI pipeline? Understand the problem Feature ideation Feature engineering Experiment Optimize Productionize What are the stages of that process that are prone to repetition? What are the ways that teams typically try to automate those steps? What are the features of generative AI models that can be brought to bear on the design stage of an AI pipeline? What are the validation/verification processes that engineers need to apply to the generated suggestions? What are the opportunities/limitations for unit/integration style tests? What are the elements of developer experience that need to be addressed to make the gen AI capabilities an enhancement instead of a distraction? What are the interfaces through which the AI functionality can/should be exposed? What are the aspects of pipeline and model deployment that can benefit from generative AI functionality? What are the potential risk factors that need to be considered when evaluating the application of this functionality? What are the most interesting, innovative, or unexpected ways that you have seen generative AI used in the development and maintenance of AI pipelines? What are the most interesting, unexpected, or challenging lessons that you have learned while working on the application of generative AI to the ML workflow? When is generative AI the wrong choice? What do you have planned for the future of FeatureByte's AI copilot capabiliteis? Contact Info LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links FeatureByte Generative AI The Art of War OCR == Optical Character Recognition Genetic Algorithm Semantic Layer Prompt Engineering The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0 Support The Machine Learning Podcast
	Learn And Automate Critical Business Workflows With 8Flow	28 Jan 2024	00:43:02
Summary Every business develops their own specific workflows to address their internal organizational needs. Not all of them are properly documented, or even visible. Workflow automation tools have tried to reduce the manual burden involved, but they are rigid and require substantial investment of time to discover and develop the routines. Boaz Hecht co-founded 8Flow to iteratively discover and automate pieces of workflows, bringing visibility and collaboration to the internal organizational processes that keep the business running. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Boaz Hecht about using AI to automate customer support at 8Flow Interview Introduction How did you get involved in machine learning? Can you describe what 8Flow is and the story behind it? How does 8Flow compare to RPA tools that companies are using today? What are the opportunities for augmenting or integrating with RPA frameworks? What are the key selling points for the solution that you are building? (does AI sell? Or is it about the realized savings?) What are the sources of signal that you are relying on to build model features? Given the heterogeneity in tools and processes across customers, what are the common focal points that let you address the widest possible range of functionality? Can you describe how 8Flow is implemented? How have the design and goals evolved since you first started working on it? What are the model categories that are most relevant for process automation in your product? How have you approached the design and implementation of your MLOps workflow? (model training, deployment, monitoring, versioning, etc.) What are the open questions around product focus and system design that you are still grappling with? Given the relative recency of ML/AI as a profession and the massive growth in attention and activity, how are you addressing the challenge of obtaining and maximizing human talent? What are the most interesting, innovative, or unexpected ways that you have seen 8Flow used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on 8Flow? When is 8Flow the wrong choice? What do you have planned for the future of 8Flow? Contact Info LinkedIn Personal Website Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links 8Flow Robotic Process Automation The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0 Support The Machine Learning Podcast
	Considering The Ethical Responsibilities Of ML And AI Engineers	28 Jan 2024	00:39:27
Summary Machine learning and AI applications hold the promise of drastically impacting every aspect of modern life. With that potential for profound change comes a responsibility for the creators of the technology to account for the ramifications of their work. In this episode Nicholas Cifuentes-Goodbody guides us through the minefields of social, technical, and ethical considerations that are necessary to ensure that this next generation of technical and economic systems are equitable and beneficial for the people that they impact. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Nicholas Cifuentes-Goodbody about the different elements of the machine learning workflow where ethics need to be considered Interview Introduction How did you get involved in machine learning? To start with, who is responsible for addressing the ethical concerns around AI? What are the different ways that AI can have positive or negative outcomes from an ethical perspective? What is the role of practitioners/individual contributors in the identification and evaluation of ethical impacts of their work? What are some utilities that are helpful in identifying and addressing bias in training data? How can practitioners address challenges of equity and accessibility in the delivery of AI products? What are some of the options for reducing the energy consumption for training and serving AI? What are the most interesting, innovative, or unexpected ways that you have seen ML teams incorporate ethics into their work? What are the most interesting, unexpected, or challenging lessons that you have learned while working on ethical implications of ML? What are some of the resources that you recommend for people who want to invest in their knowledge and application of ethics in the realm of ML? Contact Info WorldQuant University's Applied Data Science Lab LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links UNESCO Recommendation on the Ethics of Artificial Intelligence European Union AI Act How machine learning helps advance access to human rights information Disinformation, Team Jorge China, AI, and Human Rights How China Is Using A.I. to Profile a Minority Weapons of Math Destruction Fairlearn AI Fairness 360 Allen Institute for AI NYT Allen Institute for AI Transformers AI4ALL WorldQuant University How to Make Generative AI Greener Machine Learning Emissions Calculator Practicing Trustworthy Machine Learning Energy and Policy Considerations for Deep Learning Natural Language Processing Trolley Problem Protected Classes fairlearn (scikit-learn) BERT Model The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Harnessing The Engine Of AI	16 Dec 2024	00:55:13
Summary In this episode of the AI Engineering Podcast Ron Green, co-founder and CTO of KungFu AI, talks about the evolving landscape of AI systems and the challenges of harnessing generative AI engines. Ron shares his insights on the limitations of large language models (LLMs) as standalone solutions and emphasizes the need for human oversight, multi-agent systems, and robust data management to support AI initiatives. He discusses the potential of domain-specific AI solutions, RAG approaches, and mixture of experts to enhance AI capabilities while addressing risks. The conversation also explores the evolving AI ecosystem, including tooling and frameworks, strategic planning, and the importance of interpretability and control in AI systems. Ron expresses optimism about the future of AI, predicting significant advancements in the next 20 years and the integration of AI capabilities into everyday software applications. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Seamless data integration into AI applications often falls short, leading many to adopt RAG methods, which come with high costs, complexity, and limited scalability. Cognee offers a better solution with its open-source semantic memory engine that automates data ingestion and storage, creating dynamic knowledge graphs from your data. Cognee enables AI agents to understand the meaning of your data, resulting in accurate responses at a lower cost. Take full control of your data in LLM apps without unnecessary overhead. Visit aiengineeringpodcast.com/cognee to learn more and elevate your AI apps and agents. Your host is Tobias Macey and today I'm interviewing Ron Green about the wheels that we need for harnessing the power of the generative AI engine Interview Introduction How did you get involved in machine learning? Can you describe what you see as the main shortcomings of LLMs as a stand-alone solution (to anything)? The most established vehicle for harnessing LLM capabilities is the RAG pattern. What are the main limitations of that as a "product" solution? The idea of multi-agent or mixture-of-experts systems is a more sophisticated approach that is gaining some attention. What do you see as the pro/con conversation around that pattern? Beyond the system patterns that are being developed there is also a rapidly shifting ecosystem of frameworks, tools, and point solutions that plugin to various points of the AI lifecycle. How does that volatility hinder the adoption of generative AI in different contexts? In addition to the tooling, the models themselves are rapidly changing. How much does that influence the ways that organizations are thinking about whether and when to test the waters of AI? Continuing on the metaphor of LLMs and engines and the need for vehicles, where are we on the timeline in relation to the model T Ford? What are the vehicle categories that we still need to design and develop? (e.g. sedans, mini-vans, freight trucks, etc.) The current transformer architecture is starting to reach scaling limits that lead to diminishing returns. Given your perspective as an industry veteran, what are your thoughts on the future trajectory of AI model architectures? What is the ongoing role of regression style ML in the landscape of generative AI? What are the most interesting, innovative, or unexpected ways that you have seen LLMs used to power a "vehicle"? What are the most interesting, unexpected, or challenging lessons that you have learned while working in this phase of AI? When is generative AI/LLMs the wrong choice? Contact Info LinkedIn Parting Question From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links Kungfu.ai Llama open generative AI models ChatGPT Copilot Cursor RAG == Retrieval Augmented Generation Podcast Episode Mixture of Experts Deep Learning Random Forest Supervised Learning Active Learning) Yann LeCunn RLHF == Reinforcement Learning from Human Feedback Model T Ford Mamba selective state space Liquid Network Chain of thought OpenAI o1 Marvin Minsky Von Neumann Architecture Attention Is All You Need Multilayer Perceptron Dot Product Diffusion Model Gaussian Noise AlphaFold 3 Anthropic Sparse Autoencoder The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Build Intelligent Applications Faster With RelationalAI	31 Dec 2023	00:58:25
Summary Building machine learning systems and other intelligent applications are a complex undertaking. This often requires retrieving data from a warehouse engine, adding an extra barrier to every workflow. The RelationalAI engine was built as a co-processor for your data warehouse that adds a greater degree of flexibility in the representation and analysis of the underlying information, simplifying the work involved. In this episode CEO Molham Aref explains how RelationalAI is designed, the capabilities that it adds to your data clouds, and how you can start using it to build more sophisticated applications on your data. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Molham Aref about RelationalAI and the principles behind it for powering intelligent applications Interview Introduction How did you get involved in machine learning? Can you describe what RelationalAI is and the story behind it? On your site you call your product an "AI Co-processor". Can you explain what you mean by that phrase? What are the primary use cases that you address with the RelationalAI product? What are the types of solutions that teams might build to address those problems in the absence of something like the RelationalAI engine? Can you describe the system design of RelationalAI? How have the design and goals of the platform changed since you first started working on it? For someone who is using RelationalAI to address a business need, what does the onboarding and implementation workflow look like? What is your design philosophy for identifying the balance between automating the implementation of certain categories of application (e.g. NER) vs. providing building blocks and letting teams assemble them on their own? What are the data modeling paradigms that teams should be aware of to make the best use of the RKGS platform and Rel language? What are the aspects of customer education that you find yourself spending the most time on? What are some of the most under-utilized or misunderstood capabilities of the RelationalAI platform that you think deserve more attention? What are the most interesting, innovative, or unexpected ways that you have seen the RelationalAI product used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on RelationalAI? When is RelationalAI the wrong choice? What do you have planned for the future of RelationalAI? Contact Info LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links RelationalAI Snowflake AI Winter BigQuery Gradient Descent B-Tree Navigational Database Hadoop Teradata Worst Case Optimal Join Semantic Query Optimization Relational Algebra HyperGraph Linear Algebra Vector Database Pathway Data Engineering Podcast Episode Pinecone Data Engineering Podcast Episode The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Building Better AI While Preserving User Privacy With TripleBlind	22 Nov 2023	00:46:54
Summary Machine learning and generative AI systems have produced truly impressive capabilities. Unfortunately, many of these applications are not designed with the privacy of end-users in mind. TripleBlind is a platform focused on embedding privacy preserving techniques in the machine learning process to produce more user-friendly AI products. In this episode Gharib Gharibi explains how the current generation of applications can be susceptible to leaking user data and how to counteract those trends. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Gharib Gharibi about the challenges of bias and data privacy in generative AI models Interview Introduction How did you get involved in machine learning? Generative AI has been gaining a lot of attention and speculation about its impact. What are some of the risks that these capabilities pose? What are the main contributing factors to their existing shortcomings? What are some of the subtle ways that bias in the source data can manifest? In addition to inaccurate results, there is also a question of how user interactions might be re-purposed and potential impacts on data and personal privacy. What are the main sources of risk? With the massive attention that generative AI has created and the perspectives that are being shaped by it, how do you see that impacting the general perception of other implementations of AI/ML? How can ML practitioners improve and convey the trustworthiness of their models to end users? What are the risks for the industry if generative models fall out of favor with the public? How does your work at Tripleblind help to encourage a conscientious approach to AI? What are the most interesting, innovative, or unexpected ways that you have seen data privacy addressed in AI applications? What are the most interesting, unexpected, or challenging lessons that you have learned while working on privacy in AI? When is TripleBlind the wrong choice? What do you have planned for the future of TripleBlind? Contact Info LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links TripleBlind ImageNet Geoffrey Hinton Paper BERT language model Generative AI GPT == Generative Pre-trained Transformer HIPAA Safe Harbor Rules Federated Learning Differential Privacy Homomorphic Encryption The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Enhancing The Abilities Of Software Engineers With Generative AI At Tabnine	13 Nov 2023	01:04:48
Summary Software development involves an interesting balance of creativity and repetition of patterns. Generative AI has accelerated the ability of developer tools to provide useful suggestions that speed up the work of engineers. Tabnine is one of the main platforms offering an AI powered assistant for software engineers. In this episode Eran Yahav shares the journey that he has taken in building this product and the ways that it enhances the ability of humans to get their work done, and when the humans have to adapt to the tool. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Eran Yahav about building an AI powered developer assistant at Tabnine Interview Introduction How did you get involved in machine learning? Can you describe what Tabnine is and the story behind it? What are the individual and organizational motivations for using AI to generate code? What are the real-world limitations of generative AI for creating software? (e.g. size/complexity of the outputs, naming conventions, etc.) What are the elements of skepticism/oversight that developers need to exercise while using a system like Tabnine? What are some of the primary ways that developers interact with Tabnine during their development workflow? Are there any particular styles of software for which an AI is more appropriate/capable? (e.g. webapps vs. data pipelines vs. exploratory analysis, etc.) For natural languages there is a strong bias toward English in the current generation of LLMs. How does that translate into computer languages? (e.g. Python, Java, C++, etc.) Can you describe the structure and implementation of Tabnine? Do you rely primarily on a single core model, or do you have multiple models with subspecialization? How have the design and goals of the product changed since you first started working on it? What are the biggest challenges in building a custom LLM for code? What are the opportunities for specialization of the model architecture given the highly structured nature of the problem domain? For users of Tabnine, how do you assess/monitor the accuracy of recommendations? What are the feedback and reinforcement mechanisms for the model(s)? What are the most interesting, innovative, or unexpected ways that you have seen Tabnine's LLM powered coding assistant used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on AI assisted development at Tabnine? When is an AI developer assistant the wrong choice? What do you have planned for the future of Tabnine? Contact Info LinkedIn Website Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links TabNine Technion University Program Synthesis Context Stuffing Elixir Dependency Injection COBOL Verilog MidJourney The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Validating Machine Learning Systems For Safety Critical Applications With Ketryx	08 Nov 2023	00:51:12
Summary Software systems power much of the modern world. For applications that impact the safety and well-being of people there is an extra set of precautions that need to be addressed before deploying to production. If machine learning and AI are part of that application then there is a greater need to validate the proper functionality of the models. In this episode Erez Kaminski shares the work that he is doing at Ketryx to make that validation easier to implement and incorporate into the ongoing maintenance of software and machine learning products. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Erez Kaminski about using machine learning in safety critical and highly regulated medical applications Interview Introduction How did you get involved in machine learning? Can you start by describing some of the regulatory burdens placed on ML teams who are building solutions for medical applications? How do these requirements impact the development and validation processes of model design and development? What are some examples of the procedural and record-keeping aspects of the machine learning workflow that are required for FDA compliance? What are the opportunities for automating pieces of that overhead? Can you describe what you are doing at Ketryx to streamline the development/training/deployment of ML/AI applications for medical use cases? What are the ideas/assumptions that you had at the start of Ketryx that have been challenged/updated as you work with customers? What are the most interesting, innovative, or unexpected ways that you have seen ML used in medical applications? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Ketryx? When is Ketryx the wrong choice? What do you have planned for the future of Ketryx? Contact Info Email LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links Ketryx Wolfram Alpha Mathematica Tensorflow SBOM == Software Bill Of Materials Air-gapped Systems AlexNet Shapley Values SHAP Podcast.__init__ Episode Bayesian Statistics Causal Modeling Prophet FDA Principles Of Software Validation The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Applying Declarative ML Techniques To Large Language Models For Better Results	24 Oct 2023	00:46:11
Summary Large language models have gained a substantial amount of attention in the area of AI and machine learning. While they are impressive, there are many applications where they are not the best option. In this episode Piero Molino explains how declarative ML approaches allow you to make the best use of the available tools across use cases and data formats. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Piero Molino about the application of declarative ML in a world being dominated by large language models Interview Introduction How did you get involved in machine learning? Can you start by summarizing your perspective on the effect that LLMs are having on the AI/ML industry? In a world where LLMs are being applied to a growing variety of use cases, what are the capabilities that they still lack? How does declarative ML help to address those shortcomings? The majority of current hype is about commercial models (e.g. GPT-4). Can you summarize the current state of the ecosystem for open source LLMs? For teams who are investing in ML/AI capabilities, what are the sources of platform risk for LLMs? What are the comparative benefits of using a declarative ML approach? What are the most interesting, innovative, or unexpected ways that you have seen LLMs used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on declarative ML in the age of LLMs? When is an LLM the wrong choice? What do you have planned for the future of declarative ML and Predibase? Contact Info LinkedIn Website Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Links Predibase Podcast Episode Ludwig Podcast.__init__ Episode Recommender Systems Information Retrieval Vector Database Transformer Model BERT Context Windows LLAMA The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Surveying The Landscape Of AI and ML From An Investor's Perspective	15 Oct 2023	01:02:34
Summary Artificial Intelligence is experiencing a renaissance in the wake of breakthrough natural language models. With new businesses sprouting up to address the various needs of ML and AI teams across the industry, it is a constant challenge to stay informed. Matt Turck has been compiling a report on the state of ML, AI, and Data for his work at FirstMark Capital. In this episode he shares his findings on the ML and AI landscape and the interesting trends that are developing. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. As more people start using AI for projects, two things are clear: It’s a rapidly advancing field, but it’s tough to navigate. How can you get the best results for your use case? Instead of being subjected to a bunch of buzzword bingo, hear directly from pioneers in the developer and data science space on how they use graph tech to build AI-powered apps. . Attend the dev and ML talks at NODES 2023, a free online conference on October 26 featuring some of the brightest minds in tech. Check out the agenda and register today at Neo4j.com/NODES. Your host is Tobias Macey and today I'm interviewing Matt Turck about his work on the MAD (ML, AI, and Data) landscape and the insights he has gained on the ML ecosystem Interview Introduction How did you get involved in machine learning? Can you describe what the MAD landscape project is and the story behind it? What are the major changes in the ML ecosystem that you have seen since you first started compiling the landscape? How have the developments in consumer-grade AI in recent years changed the business opportunities for ML/AI? What are the coarse divisions that you see as the boundaries that define the different categories for ML/AI in the landscape? For ML infrastructure products/companies, what are the biggest challenges that they face in engineering and customer acquisition? What are some of the challenges in building momentum for startups in AI (existing moats around data access, talent acquisition, etc.)? For products/companies that have ML/AI as their core offering, what are some strategies that they use to compete with "big tech" companies that already have a large corpus of data? What do you see as the societal vs. business importance of open source models as AI becomes more integrated into consumer facing products? What are the most interesting, innovative, or unexpected ways that you have seen ML/AI used in business and social contexts? What are the most interesting, unexpected, or challenging lessons that you have learned while working on the ML/AI elements of the MAD landscape? When is ML/AI the wrong choice for businesses? What are the areas of ML/AI that you are paying closest attention to in your own work? Contact Info Website @mattturck on Twitter Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links MAD Landscape Data Engineering Podcast Episode First Mark Capital Bayesian Techniques Hadoop ChatGPT AutoGPT Dataiku Generative AI Databricks MLOps OpenAI Anthropic DeepMind BloombergGPT HuggingFace Jexi Movie "Her" Movie Synthesia The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Applying Federated Machine Learning To Sensitive Healthcare Data At Rhino Health	11 Sep 2023	00:49:54
Summary A core challenge of machine learning systems is getting access to quality data. This often means centralizing information in a single system, but that is impractical in highly regulated industries, such as healthchare. To address this hurdle Rhino Health is building a platform for federated learning on health data, so that everyone can maintain data privacy while benefiting from AI capabilities. In this episode Ittai Dayan explains the barriers to ML in healthcare and how they have designed the Rhino platform to overcome them. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Ittai Dayan about using federated learning at Rhino Health to bring AI capabilities to the tightly regulated healthcare industry Interview Introduction How did you get involved in machine learning? Can you describe what Rhino Health is and the story behind it? What is federated learning and what are the trade-offs that it introduces? What are the benefits to healthcare and pharmalogical organizations from using federated learning? What are some of the challenges that you face in validating that patient data is properly de-identified in the federated models? Can you describe what the Rhino Health platform offers and how it is implemented? How have the design and goals of the system changed since you started working on it? What are the technological capabilities that are needed for an organization to be able to start using Rhino Health to gain insights into their patient and clinical data? How have you approached the design of your product to reduce the effort to onboard new customers and solutions? What are some examples of the types of automation that you are able to provide to your customers? (e.g. medical diagnosis, radiology review, health outcome predictions, etc.) What are the ethical and regulatory challenges that you have had to address in the development of your platform? What are the most interesting, innovative, or unexpected ways that you have seen Rhino Health used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Rhino Health? When is Rhino Health the wrong choice? What do you have planned for the future of Rhino Health? Contact Info LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Rhino Health Federated Learning Nvidia Clara Nvidia DGX Melloddy Flair NLP The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Using Machine Learning To Keep An Eye On The Planet	17 Jun 2023	00:42:33
Summary Satellite imagery has given us a new perspective on our world, but it is limited by the field of view for the cameras. Synthetic Aperture Radar (SAR) allows for collecting images through clouds and in the dark, giving us a more consistent means of collecting data. In order to identify interesting details in such a vast amount of data it is necessary to use the power of machine learning. ICEYE has a fleet of satellites continuously collecting information about our planet. In this episode Tapio Friberg shares how they are applying ML to that data set to provide useful insights about fires, floods, and other terrestrial phenomena. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Tapio Friberg about building machine learning applications on top of SAR (Synthetic Aperture Radar) data to generate insights about our planet Interview Introduction How did you get involved in machine learning? Can you describe what ICEYE is and the story behind it? What are some of the applications of ML at ICEYE? What are some of the ways that SAR data poses a unique challenge to ML applications? What are some of the elements of the ML workflow that you are able to use "off the shelf" and where are the areas that you have had to build custom solutions? Can you share the structure of your engineering team and the role that the ML function plays in the larger organization? What does the end-to-end workflow for your ML model development and deployment look like? What are the operational requirements for your models? (e.g. batch execution, real-time, interactive inference, etc.) In the model definitions, what are the elements of the source domain that create the largest challenges? (e.g. noise from backscatter, variance in resolution, etc.) Once you have an output from an ML model how do you manage mapping between data domains to reflect insights from SAR sources onto a human understandable representation? Given that SAR data and earth imaging is still a very niche domain, how does that influence your ability to hire for open positions and the ways that you think about your contributions to the overall ML ecosystem? How can your work on using SAR as a representation of physical attributes help to improve capabilities in e.g. LIDAR, computer vision, etc.? What are the most interesting, innovative, or unexpected ways that you have seen ICEYE and SAR data used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on ML for SAR data? What do you have planned for the future of ML applications at ICEYE? Contact Info LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links ICEYE SAR == Synthetic Aperture Radar Transfer Learning The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	The Role Of Model Development In Machine Learning Systems	29 May 2023	00:46:41
Summary The focus of machine learning projects has long been the model that is built in the process. As AI powered applications grow in popularity and power, the model is just the beginning. In this episode Josh Tobin shares his experience from his time as a machine learning researcher up to his current work as a founder at Gantry, and the shift in focus from model development to machine learning systems. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Josh Tobin about the state of industry best practices for designing and building ML models Interview Introduction How did you get involved in machine learning? Can you start by describing what a "traditional" process for building a model looks like? What are the forces that shaped those "best practices"? What are some of the practices that are still necessary/useful and what is becoming outdated? What are the changes in the ecosystem (tooling, research, communal knowledge, etc.) that are forcing teams to reconsider how they think about modeling? What are the most critical practices/capabilities for teams who are building services powered by ML/AI? What systems do they need to support them in those efforts? Can you describe what you are building at Gantry and how it aids in the process of developing/deploying/maintaining models with "modern" workflows? What are the most challenging aspects of building a platform that supports ML teams in their workflows? What are the most interesting, innovative, or unexpected ways that you have seen teams approach model development/validation? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Gantry? When is Gantry the wrong choice? What are some of the resources that you find most helpful to stay apprised of how modeling and ML practices are evolving? Contact Info LinkedIn Website Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Gantry Full Stack Deep Learning OpenAI Kaggle NeurIPS == Neural Information Processing Systems Conference Caffe Theano Deep Learning Regression Model scikit-learn Large Language Model Foundation Models Cohere Federated Learning Feature Store dbt The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Real-Time Machine Learning Has Entered The Realm Of The Possible	09 Mar 2023	00:34:30
Summary Machine learning models have predominantly been built and updated in a batch modality. While this is operationally simpler, it doesn't always provide the best experience or capabilities for end users of the model. Tecton has been investing in the infrastructure and workflows that enable building and updating ML models with real-time data to allow you to react to real-world events as they happen. In this episode CTO Kevin Stumpf explores they benefits of real-time machine learning and the systems that are necessary to support the development and maintenance of those models. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Kevin Stumpf about the challenges and promise of real-time ML applications Interview Introduction How did you get involved in machine learning? Can you describe what real-time ML is and some examples of where it might be applied? What are the operational and organizational requirements for being able to adopt real-time approaches for ML projects? What are some of the ways that real-time requirements influence the scale/scope/architecture of an ML model? What are some of the failure modes for real-time vs analytical or operational ML? Given the low latency between source/input data being generated or received and a prediction being generated, how does that influence susceptibility to e.g. data drift? Data quality and accuracy also become more critical. What are some of the validation strategies that teams need to consider as they move to real-time? What are the most interesting, innovative, or unexpected ways that you have seen real-time ML applied? What are the most interesting, unexpected, or challenging lessons that you have learned while working on real-time ML systems? When is real-time the wrong choice for ML? What do you have planned for the future of real-time support for ML in Tecton? Contact Info LinkedIn @kevinmstumpf on Twitter Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Tecton Podcast Episode Data Engineering Podcast Episode Uber Michelangelo Reinforcement Learning Online Learning Random Forest ChatGPT XGBoost Linear Regression Train-Serve Skew Flink Data Engineering Podcast Episode The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	The Complex World of Generative AI Governance	01 Dec 2024	00:54:19
Summary In this episode of the AI Engineering Podcast Jim Olsen, CTO of ModelOp, talks about the governance of generative AI models and applications. Jim shares his extensive experience in software engineering and machine learning, highlighting the importance of governance in high-risk applications like healthcare. He explains that governance is more about the use cases of AI models rather than the models themselves, emphasizing the need for proper inventory and monitoring to ensure compliance and mitigate risks. The conversation covers challenges organizations face in implementing AI governance policies, the importance of technical controls for data governance, and the need for ongoing monitoring and baselines to detect issues like PII disclosure and model drift. Jim also discusses the balance between innovation and regulation, particularly with evolving regulations like those in the EU, and provides valuable perspectives on the current state of AI governance and the need for robust model lifecycle management. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Your host is Tobias Macey and today I'm interviewing Jim Olsen about governance of your generative AI models and applications Interview Introduction How did you get involved in machine learning? Can you describe what governance means in the context of generative AI models? (e.g. governing the models, their applications, their outputs, etc.) Governance is typically a hybrid endeavor of technical and organizational policy creation and enforcement. From the organizational perspective, what are some of the difficulties that teams are facing in understanding what those policies need to encompass? How much familiarity with the capabilities and limitations of the models is necessary to engage productively with policy debates? The regulatory landscape around AI is still very nascent. Can you give an overview of the current state of legal burden related to AI? What are some of the regulations that you consider necessary but as-of-yet absent? Data governance as a practice typically relates to controls over who can access what information and how it can be used. The controls for those policies are generally available in the data warehouse, business intelligence, etc. What are the different dimensions of technical controls that are needed in the application of generative AI systems? How much of the controls that are present for governance of analytical systems are applicable to the generative AI arena? What are the elements of risk that change when considering internal vs. consumer facing applications of generative AI? How do the modalities of the AI models impact the types of risk that are involved? (e.g. language vs. vision vs. audio) What are some of the technical aspects of the AI tools ecosystem that are in greatest need of investment to ease the burden of risk and validation of model use? What are the most interesting, innovative, or unexpected ways that you have seen AI governance implemented? What are the most interesting, unexpected, or challenging lessons that you have learned while working on AI governance? What are the technical, social, and organizational trends of AI risk and governance that you are monitoring? Contact Info LinkedIn Parting Question From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links ModelOp Foundation Models GDPR EU AI Regulation Llama 2 AWS Bedrock Shadow IT RAG == Retrieval Augmented Generation Podcast Episode Nvidia NEMO LangChain Shapley Values Gibberish Detection The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	How Shopify Built A Machine Learning Platform That Encourages Experimentation	02 Feb 2023	01:06:12
Summary Shopify uses machine learning to power multiple features in their platform. In order to reduce the amount of effort required to develop and deploy models they have invested in building an opinionated platform for their engineers. They have gone through multiple iterations of the platform and their most recent version is called Merlin. In this episode Isaac Vidas shares the use cases that they are optimizing for, how it integrates into the rest of their data platform, and how they have designed it to let machine learning engineers experiment freely and safely. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Isaac Vidas about his work on the ML platform used by Shopify Interview Introduction How did you get involved in machine learning? Can you describe what Shopify is and some of the ways that you are using ML at Shopify? What are the challenges that you have encountered as an organization in applying ML to your business needs? Can you describe how you have designed your current technical platform for supporting ML workloads? Who are the target personas for this platform? What does the workflow look like for a given data scientist/ML engineer/etc.? What are the capabilities that you are trying to optimize for in your current platform? What are some of the previous iterations of ML infrastructure and process that you have built? What are the most useful lessons that you gathered from those previous experiences that informed your current approach? How have the capabilities of the Merlin platform influenced the ways that ML is viewed and applied across Shopify? What are the most interesting, innovative, or unexpected ways that you have seen Merlin used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Merlin? When is Merlin the wrong choice? What do you have planned for the future of Merlin? Contact Info @kazuaros on Twitter LinkedIn kazuar on GitHub Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Shopify Shopify Merlin Vertex AI scikit-learn XGBoost Ray Podcast.__init__ Episode PySpark GPT-3 ChatGPT Google AI PyTorch Podcast.__init__ Episode Dask Modin Podcast.__init__ Episode Flink Data Engineering Podcast Episode Feast Feature Store Kubernetes The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Applying Machine Learning To The Problem Of Bad Data At Anomalo	24 Jan 2023	00:59:24
Summary All data systems are subject to the "garbage in, garbage out" problem. For machine learning applications bad data can lead to unreliable models and unpredictable results. Anomalo is a product designed to alert on bad data by applying machine learning models to various storage and processing systems. In this episode Jeremy Stanley discusses the various challenges that are involved in building useful and reliable machine learning models with unreliable data and the interesting problems that they are solving in the process. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Your host is Tobias Macey and today I'm interviewing Jeremy Stanley about his work at Anomalo, applying ML to the problem of data quality monitoring Interview Introduction How did you get involved in machine learning? Can you describe what Anomalo is and the story behind it? What are some of the ML approaches that you are using to address challenges with data quality/observability? What are some of the difficulties posed by your application of ML technologies on data sets that you don't control? How does the scale and quality of data that you are working with influence/constrain the algorithmic approaches that you are using to build and train your models? How have you implemented the infrastructure and workflows that you are using to support your ML applications? What are some of the ways that you are addressing data quality challenges in your own platform? What are the opportunities that you have for dogfooding your product? What are the most interesting, innovative, or unexpected ways that you have seen Anomalo used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Anomalo? When is Anomalo the wrong choice? What do you have planned for the future of Anomalo? Contact Info @jeremystan on Twitter LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Anomalo Data Engineering Podcast Episode Partial Differential Equations Neural Network Neural Networks For Pattern Recognition by Christopher M. Bishop (affiliate link) Gradient Boosted Decision Trees Shapley Values Sentry dbt Altair The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Build More Reliable Machine Learning Systems With The Dagster Orchestration Engine	02 Dec 2022	00:45:43
Summary Building a machine learning model one time can be done in an ad-hoc manner, but if you ever want to update it and serve it in production you need a way of repeating a complex sequence of operations. Dagster is an orchestration engine that understands the data that it is manipulating so that you can move beyond coarse task-based representations of your dependencies. In this episode Sandy Ryza explains how his background in machine learning has informed his work on the Dagster project and the foundational principles that it is built on to allow for collaboration across data engineering and machine learning concerns. Interview Introduction How did you get involved in machine learning? Can you start by sharing a definition of "orchestration" in the context of machine learning projects? What is your assessment of the state of the orchestration ecosystem as it pertains to ML? modeling cycles and managing experiment iterations in the execution graph how to balance flexibility with repeatability What are the most interesting, innovative, or unexpected ways that you have seen orchestration implemented/applied for machine learning? What are the most interesting, unexpected, or challenging lessons that you have learned while working on orchestration of ML workflows? When is Dagster the wrong choice? What do you have planned for the future of ML support in Dagster? Contact Info LinkedIn @s_ryz on Twitter sryza on GitHub Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Dagster Data Engineering Podcast Episode Cloudera Hadoop Apache Spark Peter Norvig Josh Wills REPL == Read Eval Print Loop RStudio Memoization MLFlow Kedro Data Engineering Podcast Episode Metaflow Podcast.__init__ Episode Kubeflow dbt Data Engineering Podcast Episode Airbyte Data Engineering Podcast Episode The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Solve The Cold Start Problem For Machine Learning By Letting Humans Teach The Computer With Aitomatic	28 Sep 2022	00:52:07
Summary Machine learning is a data-hungry approach to problem solving. Unfortunately, there are a number of problems that would benefit from the automation provided by artificial intelligence capabilities that don’t come with troves of data to build from. Christopher Nguyen and his team at Aitomatic are working to address the "cold start" problem for ML by letting humans generate models by sharing their expertise through natural language. In this episode he explains how that works, the various ways that we can start to layer machine learning capabilities on top of each other, as well as the risks involved in doing so without incorporating lessons learned in the growth of the software industry. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Predibase is a low-code ML platform without low-code limits. Built on top of our open source foundations of Ludwig and Horovod, our platform allows you to train state-of-the-art ML and deep learning models on your datasets at scale. Our platform works on text, images, tabular, audio and multi-modal data using our novel compositional model architecture. We allow users to operationalize models on top of the modern data stack, through REST and PQL – an extension of SQL that puts predictive power in the hands of data practitioners. Go to themachinelearningpodcast.com/predibase today to learn more and try it out! Your host is Tobias Macey and today I’m interviewing Christopher Nguyen about how to address the cold start problem for ML/AI projects Interview Introduction How did you get involved in machine learning? Can you describe what the "cold start" or "small data" problem is and its impact on an organization’s ability to invest in machine learning? What are some examples of use cases where ML is a viable solution but there is a corresponding lack of usable data? How does the model design influence the data requirements to build it? (e.g. statistical model vs. deep learning, etc.) What are the available options for addressing a lack of data for ML? What are the characteristics of a given data set that make it suitable for ML use cases? Can you describe what you are building at Aitomatic and how it helps to address the cold start problem? How have the design and goals of the product changed since you first started working on it? What are some of the education challenges that you face when working with organizations to help them understand how to think about ML/AI investment and practical limitations? What are the most interesting, innovative, or unexpected ways that you have seen Aitomatic/H1st used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Aitomatic/H1st? When is a human/knowledge driven approach to ML development the wrong choice? What do you have planned for the future of Aitomatic? Contact Info LinkedIn @pentagoniac on Twitter Google Scholar Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Aitomatic Human First AI Knowledge First World Symposium Atari 800 Cold start problem Scale AI Snorkel AI Podcast Episode Anomaly Detection Expert Systems ICML == International Conference on Machine Learning NIST == National Institute of Standards and Technology Multi-modal Model SVM == Support Vector Machine Tensorflow Pytorch Podcast.__init__ Episode OSS Capital DALL-E The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Convert Your Unstructured Data To Embedding Vectors For More Efficient Machine Learning With Towhee	21 Sep 2022	00:51:54
Summary Data is one of the core ingredients for machine learning, but the format in which it is understandable to humans is not a useful representation for models. Embedding vectors are a way to structure data in a way that is native to how models interpret and manipulate information. In this episode Frank Liu shares how the Towhee library simplifies the work of translating your unstructured data assets (e.g. images, audio, video, etc.) into embeddings that you can use efficiently for machine learning, and how it fits into your workflow for model development. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Building good ML models is hard, but testing them properly is even harder. At Deepchecks, they built an open-source testing framework that follows best practices, ensuring that your models behave as expected. Get started quickly using their built-in library of checks for testing and validating your model’s behavior and performance, and extend it to meet your specific needs as your model evolves. Accelerate your machine learning projects by building trust in your models and automating the testing that you used to do manually. Go to themachinelearningpodcast.com/deepchecks today to get started! Your host is Tobias Macey and today I’m interviewing Frank Liu about how to use vector embeddings in your ML projects and how Towhee can reduce the effort involved Interview Introduction How did you get involved in machine learning? Can you describe what Towhee is and the story behind it? What is the problem that Towhee is aimed at solving? What are the elements of generating vector embeddings that pose the greatest challenge or require the most effort? Once you have an embedding, what are some of the ways that it might be used in a machine learning project? Are there any design considerations that need to be addressed in the form that an embedding takes and how it impacts the resultant model that relies on it? (whether for training or inference) Can you describe how the Towhee framework is implemented? What are some of the interesting engineering challenges that needed to be addressed? How have the design/goals/scope of the project shifted since it began? What is the workflow for someone using Towhee in the context of an ML project? What are some of the types optimizations that you have incorporated into Towhee? What are some of the scaling considerations that users need to be aware of as they increase the volume or complexity of data that they are processing? What are some of the ways that using Towhee impacts the way a data scientist or ML engineer approach the design development of their model code? What are the interfaces available for integrating with and extending Towhee? What are the most interesting, innovative, or unexpected ways that you have seen Towhee used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Towhee? When is Towhee the wrong choice? What do you have planned for the future of Towhee? Contact Info LinkedIn fzliu on GitHub Website @frankzliu on Twitter Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Towhee Zilliz Milvus Data Engineering Podcast Episode Computer Vision Tensor Autoencoder Latent Space Diffusion Model HSL == Hue, Saturation, Lightness Weights and Biases The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Shedding Light On Silent Model Failures With NannyML	14 Sep 2022	01:03:18
Summary Because machine learning models are constantly interacting with inputs from the real world they are subject to a wide variety of failures. The most commonly discussed error condition is concept drift, but there are numerous other ways that things can go wrong. In this episode Wojtek Kuberski explains how NannyML is designed to compare the predicted performance of your model against its actual behavior to identify silent failures and provide context to allow you to determine whether and how urgently to address them. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Data powers machine learning, but poor data quality is the largest impediment to effective ML today. Galileo is a collaborative data bench for data scientists building Natural Language Processing (NLP) models to programmatically inspect, fix and track their data across the ML workflow (pre-training, post-training and post-production) – no more excel sheets or ad-hoc python scripts. Get meaningful gains in your model performance fast, dramatically reduce data labeling and procurement costs, while seeing 10x faster ML iterations. Galileo is offering listeners a free 30 day trial and a 30% discount on the product there after. This offer is available until Aug 31, so go to themachinelearningpodcast.com/galileo and request a demo today! Your host is Tobias Macey and today I’m interviewing Wojtek Kuberski about NannyML and the work involved in post-deployment data science Interview Introduction How did you get involved in machine learning? Can you describe what NannyML is and the story behind it? What is "post-deployment data science"? How does it differ from the metrics/monitoring approach to managing the model lifecycle? Who is typically responsible for this work? How does NannyML augment their skills? What are some of your experiences with model failure that motivated you to spend your time and focus on this problem? What are the main contributing factors to alert fatigue for ML systems? What are some of the ways that a model can fail silently? How does NannyML detect those conditions? What are the remediation actions that might be necessary once an issue is detected in a model? Can you describe how NannyML is implemented? What are some of the technical and UX design problems that you have had to address? What are some of the ideas/assumptions that you have had to re-evaluate in the process of building NannyML? What additional capabilities are necessary for supporting less structured data? Can you describe what is involved in setting up NannyML and how it fits into an ML engineer’s workflow? Once a model is deployed, what additional outputs/data can/should be collected to improve the utility of NannyML and feed into analysis of the real-world operation? What are the most interesting, innovative, or unexpected ways that you have seen NannyML used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on NannyML? When is NannyML the wrong choice? What do you have planned for the future of NannyML? Contact Info LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links NannyML F1 Score ROC Curve Concept Drift A/B Testing Jupyter Notebook Vector Embedding Airflow EDA == Exploratory Data Analysis Inspired book (affiliate link) ZenML The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	How To Design And Build Machine Learning Systems For Reasonable Scale	10 Sep 2022	00:54:10
Summary Using machine learning in production requires a sophisticated set of cooperating technologies. A majority of resources that are available for understanding how to design and operate these platforms are focused on either simple examples that don’t scale, or over-engineered technologies designed for the massive scale of big tech companies. In this episode Jacopo Tagliabue shares his vision for "ML at reasonable scale" and how you can adopt these patterns for building your own platforms. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Do you wish you could use artificial intelligence to drive your business the way Big Tech does, but don’t have a money printer? Graft is a cloud-native platform that aims to make the AI of the 1% accessible to the 99%. Wield the most advanced techniques for unlocking the value of data, including text, images, video, audio, and graphs. No machine learning skills required, no team to hire, and no infrastructure to build or maintain. For more information on Graft or to schedule a demo, visit themachinelearningpodcast.com/graft today and tell them Tobias sent you. Your host is Tobias Macey and today I’m interviewing Jacopo Tagliabue about building "reasonable scale" ML systems Interview Introduction How did you get involved in machine learning? How would you describe the current state of the ecosystem for ML practitioners? (e.g. tool selection, availability of information/tutorials, etc.) What are some of the notable changes that you have seen over the past 2 – 5 years? How have the evolutions in the data engineering space been reflected in/influenced the way that ML is being done? What are the challenges/points of friction that ML practitioners have to contend with when trying to get a model into production that isn’t just a toy? You wrote a set of tutorials and accompanying code about performing ML at "reasonable scale". What are you aiming to represent with that phrasing? There is a paradox of choice for any newcomer to ML. What are some of the key capabilities that practitioners should use in their decision rubric when designing a "reasonable scale" system? What are some of the common bottlenecks that crop up when moving from an initial test implementation to a scalable deployment that is serving customer traffic? How much of an impact does the type of ML problem being addressed have on the deployment and scalability elements of the system design? (e.g. NLP vs. computer vision vs. recommender system, etc.) What are some of the misleading pieces of advice that you have seen from "big tech" tutorials about how to do ML that are unnecessary when running at smaller scales? You also spend some time discussing the benefits of a "NoOps" approach to ML deployment. At what point do operations/infrastructure engineers need to get involved? What are the operational aspects of ML applications that infrastructure engineers working in product teams might be unprepared for? What are the most interesting, innovative, or unexpected system designs that you have seen for moderate scale MLOps? What are the most interesting, unexpected, or challenging lessons that you have learned while working on ML system design and implementation? What are the aspects of ML systems design that you are paying attention to in the current ecosystem? What advice do you have for additional references or research that ML practitioners would benefit from when designing their own production systems? Contact Info jacopotagliabue on GitHub Website LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links The Post-Modern Stack: ML At Reasonable Scale Coveo NLP == Natural Language Processing RecList Part of speech tagging Markov Model YDNABB (You Don’t Need A Bigger Boat) dbt Data Engineering Podcast Episode Seldon Metaflow Podcast.__init__ Episode Snowflake Information Retrieval Modern Data Stack SQLite Spark SQL AWS Athena Keras PyTorch Luigi Airflow Flask AWS Fargate AWS Sagemaker Recommendations At Reasonable Scale Pinecone Data Engineering Podcast Episode Redis KNN == K-Nearest Neighbors Pinterest Engineering Blog Materialize OpenAI The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Building A Business Powered By Machine Learning At Assembly AI	09 Sep 2022	00:58:43
Summary The increasing sophistication of machine learning has enabled dramatic transformations of businesses and introduced new product categories. At Assembly AI they are offering advanced speech recognition and natural language models as an API service. In this episode founder Dylan Fox discusses the unique challenges of building a business with machine learning as the core product. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Predibase is a low-code ML platform without low-code limits. Built on top of our open source foundations of Ludwig and Horovod, our platform allows you to train state-of-the-art ML and deep learning models on your datasets at scale. Our platform works on text, images, tabular, audio and multi-modal data using our novel compositional model architecture. We allow users to operationalize models on top of the modern data stack, through REST and PQL – an extension of SQL that puts predictive power in the hands of data practitioners. Go to themachinelearningpodcast.com/predibase today to learn more and try it out! Your host is Tobias Macey and today I’m interviewing Dylan Fox about building and growing a business with ML as its core offering Interview Introduction How did you get involved in machine learning? Can you describe what Assembly is and the story behind it? For anyone who isn’t familiar with your platform, can you describe the role that ML/AI plays in your product? What was your process for going from idea to prototype for an AI powered business? Can you offer parallels between your own experience and that of your peers who are building businesses oriented more toward pure software applications? How are you structuring your teams? On the path to your current scale and capabilities how have you managed scoping of your model capabilities and operational scale to avoid getting bogged down or burnt out? How do you think about scoping of model functionality to balance composability and system complexity? What is your process for identifying and understanding which problems are suited to ML and when to rely on pure software? You are constantly iterating on model performance and introducing new capabilities. How do you manage prototyping and experimentation cycles? What are the metrics that you track to identify whether and when to move from an experimental to an operational state with a model? What is your process for understanding what’s possible and what can feasibly operate at scale? Can you describe your overall operational patterns delivery process for ML? What are some of the most useful investments in tooling that you have made to manage development experience for your teams? Once you have a model in operation, how do you manage performance tuning? (from both a model and an operational scalability perspective) What are the most interesting, innovative, or unexpected aspects of ML development and maintenance that you have encountered while building and growing the Assembly platform? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Assembly? When is ML the wrong choice? What do you have planned for the future of Assembly? Contact Info @YouveGotFox on Twitter LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Assembly AI Podcast.__init__ Episode Learn Python the Hard Way NLTK NLP == Natural Language Processing NLU == Natural Language Understanding Speech Recognition Tensorflow r/machinelearning SciPy PyTorch Jax HuggingFace RNN == Recurrent Neural Network CNN == Convolutional Neural Network LSTM == Long Short Term Memory Hidden Markov Models Baidu DeepSpeech CTC (Connectionist Temporal Classification) Loss Model Twilio Grid Search K80 GPU A100 GPU TPU == Tensor Processing Unit Foundation Models BLOOM Language Model DALL-E 2 The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Update Your Model's View Of The World In Real Time With Streaming Machine Learning Using River	26 Aug 2022	01:15:21
Summary The majority of machine learning projects that you read about or work on are built around batch processes. The model is trained, and then validated, and then deployed, with each step being a discrete and isolated task. Unfortunately, the real world is rarely static, leading to concept drift and model failures. River is a framework for building streaming machine learning projects that can constantly adapt to new information. In this episode Max Halford explains how the project works, why you might (or might not) want to consider streaming ML, and how to get started building with River. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Building good ML models is hard, but testing them properly is even harder. At Deepchecks, they built an open-source testing framework that follows best practices, ensuring that your models behave as expected. Get started quickly using their built-in library of checks for testing and validating your model’s behavior and performance, and extend it to meet your specific needs as your model evolves. Accelerate your machine learning projects by building trust in your models and automating the testing that you used to do manually. Go to themachinelearningpodcast.com/deepchecks today to get started! Your host is Tobias Macey and today I’m interviewing Max Halford about River, a Python toolkit for streaming and online machine learning Interview Introduction How did you get involved in machine learning? Can you describe what River is and the story behind it? What is "online" machine learning? What are the practical differences with batch ML? Why is batch learning so predominant? What are the cases where someone would want/need to use online or streaming ML? The prevailing pattern for batch ML model lifecycles is to train, deploy, monitor, repeat. What does the ongoing maintenance for a streaming ML model look like? Concept drift is typically due to a discrepancy between the data used to train a model and the actual data being observed. How does the use of online learning affect the incidence of drift? Can you describe how the River framework is implemented? How have the design and goals of the project changed since you started working on it? How do the internal representations of the model differ from batch learning to allow for incremental updates to the model state? In the documentation you note the use of Python dictionaries for state management and the flexibility offered by that choice. What are the benefits and potential pitfalls of that decision? Can you describe the process of using River to design, implement, and validate a streaming ML model? What are the operational requirements for deploying and serving the model once it has been developed? What are some of the challenges that users of River might run into if they are coming from a batch learning background? What are the most interesting, innovative, or unexpected ways that you have seen River used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on River? When is River the wrong choice? What do you have planned for the future of River? Contact Info Email @halford_max on Twitter MaxHalford on GitHub Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links River scikit-multiflow Federated Machine Learning Hogwild! Google Paper Chip Huyen concept drift blog post Dan Crenshaw Berkeley Clipper MLOps Robustness Principle NY Taxi Dataset RiverTorch River Public Roadmap Beaver tool for deploying online models Prodigy ML human in the loop labeling The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Using AI To Transform Your Business Without The Headache Using Graft	16 Aug 2022	01:07:34
Summary Machine learning is a transformative tool for the organizations that can take advantage of it. While the frameworks and platforms for building machine learning applications are becoming more powerful and broadly available, there is still a significant investment of time, money, and talent required to take full advantage of it. In order to reduce that barrier further Adam Oliner and Brian Calvert, along with their other co-founders, started Graft. In this episode Adam and Brian explain how they have built a platform designed to empower everyone in the business to take part in designing and building ML projects, while managing the end-to-end workflow required to go from data to production. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Predibase is a low-code ML platform without low-code limits. Built on top of our open source foundations of Ludwig and Horovod, our platform allows you to train state-of-the-art ML and deep learning models on your datasets at scale. Our platform works on text, images, tabular, audio and multi-modal data using our novel compositional model architecture. We allow users to operationalize models on top of the modern data stack, through REST and PQL – an extension of SQL that puts predictive power in the hands of data practitioners. Go to themachinelearningpodcast.com/predibase today to learn more and try it out! Building good ML models is hard, but testing them properly is even harder. At Deepchecks, they built an open-source testing framework that follows best practices, ensuring that your models behave as expected. Get started quickly using their built-in library of checks for testing and validating your model’s behavior and performance, and extend it to meet your specific needs as your model evolves. Accelerate your machine learning projects by building trust in your models and automating the testing that you used to do manually. Go to themachinelearningpodcast.com/deepchecks today to get started! Your host is Tobias Macey and today I’m interviewing Brian Calvert and Adam Oliner about Graft, a cloud-native platform designed to simplify the work of applying AI to business problems Interview Introduction How did you get involved in machine learning? Can you describe what Graft is and the story behind it? What is the core thesis of the problem you are targeting? How does the Graft product address that problem? Who are the personas that you are focused on working with both now in your early stages and in the future as you evolve the product? What are the capabilities that can be unlocked in different organizations by reducing the friction and up-front investment required to adopt ML/AI? What are the user-facing interfaces that you are focused on providing to make that adoption curve as shallow as possible? What are some of the unavoidable bits of complexity that need to be surfaced to the end user? Can you describe the infrastructure and platform design that you are relying on for the Graft product? What are some of the emerging "best practices" around ML/AI that you have been able to build on top of? As new techniques and practices are discovered/introduced how are you thinking about the adoption process and how/when to integrate them into the Graft product? What are some of the new engineering challenges that you have had to tackle as a result of your specific product? Machine learning can be a very data and compute intensive endeavor. How are you thinking about scalability in a multi-tenant system? Different model and data types can be widely divergent in terms of the cost (monetary, time, compute, etc.) required. How are you thinking about amortizing vs. passing through those costs to the end user? Can you describe the adoption/integration process for someone using Graft? Once they are onboarded and they have connected to their various data sources, what is the workflow for someone to apply ML capabilities to their problems? One of the challenges about the current state of ML capabilities and adoption is understanding what is possible and what is impractical. How have you designed Graft to help identify and expose opportunities for applying ML within the organization? What are some of the challenges of customer education and overall messaging that you are working through? What are the most interesting, innovative, or unexpected ways that you have seen Graft used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Graft? When is Graft the wrong choice? What do you have planned for the future of Graft? Contact Info Brian LinkedIn Adam LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Graft High Energy Particle Physics LHC Cruise Slack Splunk Marvin Minsky Patrick Henry Winston AI Winter Sebastian Thrun DARPA Grand Challenge Higss Boson Supersymmetry Kinematics Transfer Learning Foundation Models ML Embeddings BERT Airflow Dagster Prefect Dask Kubeflow MySQL PostgreSQL Snowflake Redshift S3 Kubernetes Multi-modal models Multi-task models Magic: The Gathering The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/[CC BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/?utm_source=rss&utm_medium=rss
	Building Semantic Memory for AI With Cognee	25 Nov 2024	00:55:01
Summary In this episode of the AI Engineering Podcast, Vasilije Markovich talks about enhancing Large Language Models (LLMs) with memory to improve their accuracy. He discusses the concept of memory in LLMs, which involves managing context windows to enhance reasoning without the high costs of traditional training methods. He explains the challenges of forgetting in LLMs due to context window limitations and introduces the idea of hierarchical memory, where immediate retrieval and long-term information storage are balanced to improve application performance. Vasilije also shares his work on Cognee, a tool he's developing to manage semantic memory in AI systems, and discusses its potential applications beyond its core use case. He emphasizes the importance of combining cognitive science principles with data engineering to push the boundaries of AI capabilities and shares his vision for the future of AI systems, highlighting the role of personalization and the ongoing development of Cognee to support evolving AI architectures. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Your host is Tobias Macey and today I'm interviewing Vasilije Markovic about adding memory to LLMs to improve their accuracy Interview Introduction How did you get involved in machine learning? Can you describe what "memory" is in the context of LLM systems? What are the symptoms of "forgetting" that manifest when interacting with LLMs? How do these issues manifest between single-turn vs. multi-turn interactions? How does the lack of hierarchical and evolving memory limit the capabilities of LLM systems? What are the technical/architectural requirements to add memory to an LLM system/application? How does Cognee help to address the shortcomings of current LLM/RAG architectures? Can you describe how Cognee is implemented? Recognizing that it has only existed for a short time, how have the design and scope of Cognee evolved since you first started working on it? What are the data structures that are most useful for managing the memory structures? For someone who wants to incorporate Cognee into their LLM architecture, what is involved in integrating it into their applications? How does it change the way that you think about the overall requirements for an LLM application? For systems that interact with multiple LLMs, how does Cognee manage context across those systems? (e.g. different agents for different use cases) There are other systems that are being built to manage user personalization in LLm applications, how do the goals of Cognee relate to those use cases? (e.g. Mem0 - https://github.com/mem0ai/mem0) What are the unknowns that you are still navigating with Cognee? What are the most interesting, innovative, or unexpected ways that you have seen Cognee used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Cognee? When is Cognee the wrong choice? What do you have planned for the future of Cognee? Contact Info LinkedIn Parting Question From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links Cognee Montenegro Catastrophic Forgetting Multi-Turn Interaction RAG == Retrieval Augmented Generation Podcast Episode GraphRAG Podcast Episode Long-term memory Short-term memory Langchain LlamaIndex Haystack dlt Data Engineering Podcast Episode Pinecone Podcast Episode Agentic RAG Airflow DAG == Directed Acyclic Graph FalkorDB Neo4J Pydantic AWS ECS AWS SNS AWS SQS AWS Lambda LLM As Judge Mem0 QDrant LanceDB DuckDB The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Accelerate Development And Delivery Of Your Machine Learning Projects With A Comprehensive Feature Platform	06 Aug 2022	00:50:38
Summary In order for a machine learning model to build connections and context across the data that is fed into it the raw data needs to be engineered into semantic features. This is a process that can be tedious and full of toil, requiring constant upkeep and often leading to rework across projects and teams. In order to reduce the amount of wasted effort and speed up experimentation and training iterations a new generation of services are being developed. Tecton first built a feature store to serve as a central repository of engineered features and keep them up to date for training and inference. Since then they have expanded the set of tools and services to be a full-fledged feature platform. In this episode Kevin Stumpf explains the different capabilities and activities related to features that are necessary to maintain velocity in your machine learning projects. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Building good ML models is hard, but testing them properly is even harder. At Deepchecks, they built an open-source testing framework that follows best practices, ensuring that your models behave as expected. Get started quickly using their built-in library of checks for testing and validating your model’s behavior and performance, and extend it to meet your specific needs as your model evolves. Accelerate your machine learning projects by building trust in your models and automating the testing that you used to do manually. Go to themachinelearningpodcast.com/deepchecks today to get started! Do you wish you could use artificial intelligence to drive your business the way Big Tech does, but don’t have a money printer? Graft is a cloud-native platform that aims to make the AI of the 1% accessible to the 99%. Wield the most advanced techniques for unlocking the value of data, including text, images, video, audio, and graphs. No machine learning skills required, no team to hire, and no infrastructure to build or maintain. For more information on Graft or to schedule a demo, visit themachinelearningpodcast.com/graft today and tell them Tobias sent you. Data powers machine learning, but poor data quality is the largest impediment to effective ML today. Galileo is a collaborative data bench for data scientists building Natural Language Processing (NLP) models to programmatically inspect, fix and track their data across the ML workflow (pre-training, post-training and post-production) – no more excel sheets or ad-hoc python scripts. Get meaningful gains in your model performance fast, dramatically reduce data labeling and procurement costs, while seeing 10x faster ML iterations. Galileo is offering listeners a free 30 day trial and a 30% discount on the product there after. This offer is available until Aug 31, so go to themachinelearningpodcast.com/galileo and request a demo today! Your host is Tobias Macey and today I’m interviewing Kevin Stumpf about the role of feature platforms in your ML engineering workflow Interview Introduction How did you get involved in machine learning? Can you describe what you mean by the term "feature platform"? What are the components and supporting capabilities that are needed for such a platform? How does the availability of engineered features impact the ability of an organization to put ML into production? What are the points of friction that teams encounter when trying to build and maintain ML projects in the absence of a fully integrated feature platform? Who are the target personas for the Tecton platform? What stages of the ML lifecycle does it address? Can you describe how you have designed the Tecton feature platform? How have the goals and capabilities of the product evolved since you started working on it? What is the workflow for an ML engineer or data scientist to build and maintain features and use them in the model development workflow? What are the responsibilities of the MLOps stack that you have intentionally decided not to address? What are the interfaces and extension points that you offer for integrating with the other utilities needed to manage a full ML system? You wrote a post about the need to establish a DevOps approach to ML data. In keeping with that theme, can you describe how to think about the approach to testing and validation techniques for features and their outputs? What are the most interesting, innovative, or unexpected ways that you have seen Tecton/Feast used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Tecton? When is Tecton the wrong choice? What do you have planned for the future of the Tecton feature platform? Contact Info LinkedIn @kevinmstumpf on Twitter kevinstumpf on GitHub Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Links Tecton Data Engineering Podcast Episode Uber Michaelangelo Feature Store Snowflake Data Engineering Podcast Episode DynamoDB Train/Serve Skew Lambda Architecture Redis The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/[CC BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/?utm_source=rss&utm_medium=rss
	Build Better Models Through Data Centric Machine Learning Development With Snorkel AI	29 Jul 2022	00:53:49
Summary Machine learning is a data hungry activity, and the quality of the resulting model is highly dependent on the quality of the inputs that it receives. Generating sufficient quantities of high quality labeled data is an expensive and time consuming process. In order to reduce that time and cost Alex Ratner and his team at Snorkel AI have built a system for powering data-centric machine learning development. In this episode he explains how the Snorkel platform allows domain experts to create labeling functions that translate their expertise into reusable logic that dramatically reduces the time needed to build training data sets and drives down the total cost. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Building good ML models is hard, but testing them properly is even harder. At Deepchecks, they built an open-source testing framework that follows best practices, ensuring that your models behave as expected. Get started quickly using their built-in library of checks for testing and validating your model’s behavior and performance, and extend it to meet your specific needs as your model evolves. Accelerate your machine learning projects by building trust in your models and automating the testing that you used to do manually. Go to themachinelearningpodcast.com/deepchecks today to get started! Data powers machine learning, but poor data quality is the largest impediment to effective ML today. Galileo is a collaborative data bench for data scientists building Natural Language Processing (NLP) models to programmatically inspect, fix and track their data across the ML workflow (pre-training, post-training and post-production) – no more excel sheets or ad-hoc python scripts. Get meaningful gains in your model performance fast, dramatically reduce data labeling and procurement costs, while seeing 10x faster ML iterations. Galileo is offering listeners a free 30 day trial and a 30% discount on the product there after. This offer is available until Aug 31, so go to themachinelearningpodcast.com/galileo and request a demo today! Predibase is a low-code ML platform without low-code limits. Built on top of our open source foundations of Ludwig and Horovod, our platform allows you to train state-of-the-art ML and deep learning models on your datasets at scale. Our platform works on text, images, tabular, audio and multi-modal data using our novel compositional model architecture. We allow users to operationalize models on top of the modern data stack, through REST and PQL – an extension of SQL that puts predictive power in the hands of data practitioners. Go to themachinelearningpodcast.com/predibase today to learn more and try it out! Your host is Tobias Macey and today I’m interviewing Alex Ratner about Snorkel AI, a platform for data-centric machine learning workflows powered by programmatic data labeling techniques Interview Introduction How did you get involved in machine learning? Can you describe what Snorkel AI is and the story behind it? What are the problems that you are focused on solving? Which pieces of the ML lifecycle are you focused on? How did your experience building the open source Snorkel project and working with the community inform your product direction for Snorkel AI? How has the underlying Snorkel project evolved over the past 4 years? What are the deciding factors that an organization or ML team need to consider when evaluating existing labeling strategies against the programmatic approach that you provide? What are the features that Snorkel provides over and above managing code execution across the source data set? Can you describe what you have built at Snorkel AI and how it is implemented? What are some of the notable developments of the ML ecosystem that had a meaningful impact on your overall product vision/viability? Can you describe the workflow for an individual or team who is using Snorkel for generating their training data set? How does Snorkel integrate with the experimentation process to track how changes to labeling logic correlate with the performance of the resulting model? What are some of the complexities involved in designing and testing the labeling logic? How do you handle complex data formats such as audio, video, images, etc. that might require their own ML models to generate labels? (e.g. object detection for bounding boxes) With the increased scale and quality of labeled data that Snorkel AI offers, how does that impact the viability of autoML toolchains for generating useful models? How are you managing the governance and feature boundaries between the open source Snorkel project and the business that you have built around it? What are the most interesting, innovative, or unexpected ways that you have seen Snorkel AI used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Snorkel AI? When is Snorkel AI the wrong choice? What do you have planned for the future of Snorkel AI? Contact Info LinkedIn Website @ajratner on Twitter Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Snorkel AI Data Engineering Podcast Episode University of Washington Snorkel OSS Natural Language Processing (NLP) Tensorflow PyTorch Podcast.__init__ Episode Deep Learning Foundation Models MLFlow SHAP Podcast.__init__ Episode The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Declarative Machine Learning For High Performance Deep Learning Models With Predibase	21 Jul 2022	01:00:20
Summary Deep learning is a revolutionary category of machine learning that accelerates our ability to build powerful inference models. Along with that power comes a great deal of complexity in determining what neural architectures are best suited to a given task, engineering features, scaling computation, etc. Predibase is building on the successes of the Ludwig framework for declarative deep learning and Horovod for horizontally distributing model training. In this episode CTO and co-founder of Predibase, Travis Addair, explains how they are reducing the burden of model development even further with their managed service for declarative and low-code ML and how they are integrating with the growing ecosystem of solutions for the full ML lifecycle. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Building good ML models is hard, but testing them properly is even harder. At Deepchecks, they built an open-source testing framework that follows best practices, ensuring that your models behave as expected. Get started quickly using their built-in library of checks for testing and validating your model’s behavior and performance, and extend it to meet your specific needs as your model evolves. Accelerate your machine learning projects by building trust in your models and automating the testing that you used to do manually. Go to themachinelearningpodcast.com/deepchecks today to get started! Data powers machine learning, but poor data quality is the largest impediment to effective ML today. Galileo is a collaborative data bench for data scientists building Natural Language Processing (NLP) models to programmatically inspect, fix and track their data across the ML workflow (pre-training, post-training and post-production) – no more excel sheets or ad-hoc python scripts. Get meaningful gains in your model performance fast, dramatically reduce data labeling and procurement costs, while seeing 10x faster ML iterations. Galileo is offering listeners a free 30 day trial and a 30% discount on the product there after. This offer is available until Aug 31, so go to themachinelearningpodcast.com/galileo and request a demo today! Do you wish you could use artificial intelligence to drive your business the way Big Tech does, but don’t have a money printer? Graft is a cloud-native platform that aims to make the AI of the 1% accessible to the 99%. Wield the most advanced techniques for unlocking the value of data, including text, images, video, audio, and graphs. No machine learning skills required, no team to hire, and no infrastructure to build or maintain. For more information on Graft or to schedule a demo, visit themachinelearningpodcast.com/graft today and tell them Tobias sent you. Your host is Tobias Macey and today I’m interviewing Travis Addair about Predibase, a low-code platform for building ML models in a declarative format Interview Introduction How did you get involved in machine learning? Can you describe what Predibase is and the story behind it? Who is your target audience and how does that focus influence your user experience and feature development priorities? How would you describe the semantic differences between your chosen terminology of "declarative ML" and the "autoML" nomenclature that many projects and products have adopted? Another platform that launched recently with a promise of "declarative ML" is Continual. How would you characterize your relative strengths? Can you describe how the Predibase platform is implemented? How have the design and goals of the product changed as you worked through the initial implementation and started working with early customers? The operational aspects of the ML lifecycle are still fairly nascent. How have you thought about the boundaries for your product to avoid getting drawn into scope creep while providing a happy path to delivery? Ludwig is a core element of your platform. What are the other capabilities that you are layering around and on top of it to build a differentiated product? In addition to the existing interfaces for Ludwig you created a new language in the form of PQL. What was the motivation for that decision? How did you approach the semantic and syntactic design of the dialect? What is your vision for PQL in the space of "declarative ML" that you are working to define? Can you describe the available workflows for an individual or team that is using Predibase for prototyping and validating an ML model? Once a model has been deemed satisfactory, what is the path to production? How are you approaching governance and sustainability of Ludwig and Horovod while balancing your reliance on them in Predibase? What are some of the notable investments/improvements that you have made in Ludwig during your work of building Predibase? What are the most interesting, innovative, or unexpected ways that you have seen Predibase used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Predibase? When is Predibase the wrong choice? What do you have planned for the future of Predibase? Contact Info LinkedIn tgaddair on GitHub @travisaddair on Twitter Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Predibase Horovod Ludwig Podcast.__init__ Episode Support Vector Machine Hadoop Tensorflow Uber Michaelangelo AutoML Spark ML Lib Deep Learning PyTorch Continual Data Engineering Podcast Episode Overton Kubernetes Ray Nvidia Triton Whylogs Data Engineering Podcast Episode Weights and Biases MLFlow Comet Confusion Matrices dbt Data Engineering Podcast Episode Torchscript Self-supervised Learning The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Stop Feeding Garbage Data To Your ML Models, Clean It Up With Galileo	14 Jul 2022	00:47:04
Summary Machine learning is a force multiplier that can generate an outsized impact on your organization. Unfortunately, if you are feeding your ML model garbage data, then you will get orders of magnitude more garbage out of it. The team behind Galileo experienced that pain for themselves and have set out to make data management and cleaning for machine learning a first class concern in your workflow. In this episode Vikram Chatterji shares the story of how Galileo got started and how you can use their platform to fix your ML data so that you can get back to the fun parts. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Predibase is a low-code ML platform without low-code limits. Built on top of our open source foundations of Ludwig and Horovod, our platform allows you to train state-of-the-art ML and deep learning models on your datasets at scale. Our platform works on text, images, tabular, audio and multi-modal data using our novel compositional model architecture. We allow users to operationalize models on top of the modern data stack, through REST and PQL – an extension of SQL that puts predictive power in the hands of data practitioners. Go to themachinelearningpodcast.com/predibase today to learn more and try it out! Do you wish you could use artificial intelligence to drive your business the way Big Tech does, but don’t have a money printer? Graft is a cloud-native platform that aims to make the AI of the 1% accessible to the 99%. Wield the most advanced techniques for unlocking the value of data, including text, images, video, audio, and graphs. No machine learning skills required, no team to hire, and no infrastructure to build or maintain. For more information on Graft or to schedule a demo, visit themachinelearningpodcast.com/graft today and tell them Tobias sent you. Building good ML models is hard, but testing them properly is even harder. At Deepchecks, they built an open-source testing framework that follows best practices, ensuring that your models behave as expected. Get started quickly using their built-in library of checks for testing and validating your model’s behavior and performance, and extend it to meet your specific needs as your model evolves. Accelerate your machine learning projects by building trust in your models and automating the testing that you used to do manually. Go to themachinelearningpodcast.com/deepchecks today to get started! Your host is Tobias Macey and today I’m interviewing Vikram Chatterji about Galileo, a platform for uncovering and addressing data problems to improve your model quality Interview Introduction How did you get involved in machine learning? Can you describe what Galileo is and the story behind it? Who are the target users of the platform and what are the tools/workflows that you are replacing? How does that focus inform and influence the design and prioritization of features in the platform? What are some of the real-world impacts that you have experienced as a result of the kinds of data problems that you are addressing with Galileo? Can you describe how the Galileo product is implemented? What are some of the assumptions that you had formed from your own experiences that have been challenged as you worked with early design partners? The toolchains and model architectures of any given team is unlikely to be a perfect match across departments or organizations. What are the core principles/concepts that you have hooked into in order to provide the broadest compatibility? What are the model types/frameworks/etc. that you have had to forego support for in the early versions of your product? Can you describe the workflow for someone building a machine learning model and how Galileo fits across the various stages of that cycle? What are some of the biggest difficulties posed by the non-linear nature of the experimentation cycle in model development? What are some of the ways that you work to quantify the impact of your tool on the productivity and profit contributions of an ML team/organization? What are the most interesting, innovative, or unexpected ways that you have seen Galileo used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Galileo? When is Galileo the wrong choice? What do you have planned for the future of Galileo? Contact Info LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Galileo F1 Score Tensorflow Keras SpaCy Podcast.__init__ Episode Pytorch Podcast.__init__ Episode MXNet Jax The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Build Better Machine Learning Models With Confidence By Adding Validation With Deepchecks	06 Jul 2022	00:48:40
Summary Machine learning has the potential to transform industries and revolutionize business capabilities, but only if the models are reliable and robust. Because of the fundamental probabilistic nature of machine learning techniques it can be challenging to test and validate the generated models. The team at Deepchecks understands the widespread need to easily and repeatably check and verify the outputs of machine learning models and the complexity involved in making it a reality. In this episode Shir Chorev and Philip Tannor explain how they are addressing the problem with their open source deepchecks library and how you can start using it today to build trust in your machine learning applications. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Do you wish you could use artificial intelligence to drive your business the way Big Tech does, but don’t have a money printer? Graft is a cloud-native platform that aims to make the AI of the 1% accessible to the 99%. Wield the most advanced techniques for unlocking the value of data, including text, images, video, audio, and graphs. No machine learning skills required, no team to hire, and no infrastructure to build or maintain. For more information on Graft or to schedule a demo, visit themachinelearningpodcast.com/graft today and tell them Tobias sent you. Predibase is a low-code ML platform without low-code limits. Built on top of our open source foundations of Ludwig and Horovod, our platform allows you to train state-of-the-art ML and deep learning models on your datasets at scale. Our platform works on text, images, tabular, audio and multi-modal data using our novel compositional model architecture. We allow users to operationalize models on top of the modern data stack, through REST and PQL – an extension of SQL that puts predictive power in the hands of data practitioners. Go to themachinelearningpodcast.com/predibase today to learn more and try it out! Data powers machine learning, but poor data quality is the largest impediment to effective ML today. Galileo is a collaborative data bench for data scientists building Natural Language Processing (NLP) models to programmatically inspect, fix and track their data across the ML workflow (pre-training, post-training and post-production) – no more excel sheets or ad-hoc python scripts. Get meaningful gains in your model performance fast, dramatically reduce data labeling and procurement costs, while seeing 10x faster ML iterations. Galileo is offering listeners a free 30 day trial and a 30% discount on the product there after. This offer is available until Aug 31, so go to themachinelearningpodcast.com/galileo and request a demo today! Your host is Tobias Macey and today I’m interviewing Shir Chorev and Philip Tannor about Deepchecks, a Python package for comprehensively validating your machine learning models and data with minimal effort. Interview Introduction How did you get involved in machine learning? Can you describe what Deepchecks is and the story behind it? Who is the target audience for the project? What are the biggest challenges that these users face in bringing ML models from concept to production and how does DeepChecks address those problems? In the absence of DeepChecks how are practitioners solving the problems of model validation and comparison across iteratiosn? What are some of the other tools in this ecosystem and what are the differentiating features of DeepChecks? What are some examples of the kinds of tests that are useful for understanding the "correctness" of models? What are the methods by which ML engineers/data scientists/domain experts can define what "correctness" means in a given model or subject area? In software engineering the categories of tests are tiered as unit -> integration -> end-to-end. What are the relevant categories of tests that need to be built for validating the behavior of machine learning models? How do model monitoring utilities overlap with the kinds of tests that you are building with deepchecks? Can you describe how the DeepChecks package is implemented? How have the design and goals of the project changed or evolved from when you started working on it? What are the assumptions that you have built up from your own experiences that have been challenged by your early users and design partners? Can you describe the workflow for an individual or team using DeepChecks as part of their model training and deployment lifecycle? Test engineering is a deep discipline in its own right. How have you approached the user experience and API design to reduce the overhead for ML practitioners to adopt good practices? What are the interfaces available for creating reusable tests and composing test suites together? What are the additional services/capabilities that you are providing in your commercial offering? How are you managing the governance and sustainability of the OSS project and balancing that against the needs/priorities of the business? What are the most interesting, innovative, or unexpected ways that you have seen DeepChecks used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on DeepChecks? When is DeepChecks the wrong choice? What do you have planned for the future of DeepChecks? Contact Info Shir LinkedIn shir22 on GitHub Philip LinkedIn @philiptannor on Twitter Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links DeepChecks Random Forest Talpiot Program SHAP Podcast.__init__ Episode Airflow Great Expectations Data Engineering Podcast Episode The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Build A Full Stack ML Powered App In An Afternoon With Baseten	29 Jun 2022	00:46:26
Summary Building an ML model is getting easier than ever, but it is still a challenge to get that model in front of the people that you built it for. Baseten is a platform that helps you quickly generate a full stack application powered by your model. You can easily create a web interface and APIs powered by the model you created, or a pre-trained model from their library. In this episode Tuhin Srivastava, co-founder of Basten, explains how the platform empowers data scientists and ML engineers to get their work in production without having to negotiate for help from their application development colleagues. Announcements Hello and welcome to the Machine Learning Podcast, the podcast about machine learning and how to bring it from idea to delivery. Data powers machine learning, but poor data quality is the largest impediment to effective ML today. Galileo is a collaborative data bench for data scientists building Natural Language Processing (NLP) models to programmatically inspect, fix and track their data across the ML workflow (pre-training, post-training and post-production) – no more excel sheets or ad-hoc python scripts. Get meaningful gains in your model performance fast, dramatically reduce data labeling and procurement costs, while seeing 10x faster ML iterations. Galileo is offering listeners a free 30 day trial and a 30% discount on the product there after. This offer is available until Aug 31, so go to themachinelearningpodcast.com/galileo and request a demo today! Do you wish you could use artificial intelligence to drive your business the way Big Tech does, but don’t have a money printer? Graft is a cloud-native platform that aims to make the AI of the 1% accessible to the 99%. Wield the most advanced techniques for unlocking the value of data, including text, images, video, audio, and graphs. No machine learning skills required, no team to hire, and no infrastructure to build or maintain. For more information on Graft or to schedule a demo, visit themachinelearningpodcast.com/graft today and tell them Tobias sent you. Predibase is a low-code ML platform without low-code limits. Built on top of our open source foundations of Ludwig and Horovod, our platform allows you to train state-of-the-art ML and deep learning models on your datasets at scale. Our platform works on text, images, tabular, audio and multi-modal data using our novel compositional model architecture. We allow users to operationalize models on top of the modern data stack, through REST and PQL – an extension of SQL that puts predictive power in the hands of data practitioners. Go to themachinelearningpodcast.com/predibase today to learn more and try it out! Your host is Tobias Macey and today I’m interviewing Tuhin Srivastava about Baseten, an ML Application Builder for data science and machine learning teams Interview Introduction How did you get involved in machine learning? Can you describe what Baseten is and the story behind it? Who are the target users for Baseten and what problems are you solving for them? What are some of the typical technical requirements for an application that is powered by a machine learning model? In the absence of Baseten, what are some of the common utilities/patterns that teams might rely on? What kinds of challenges do teams run into when serving a model in the context of an application? There are a number of projects that aim to reduce the overhead of turning a model into a usable product (e.g. Streamlit, Hex, etc.). What is your assessment of the current ecosystem for lowering the barrier to product development for ML and data science teams? Can you describe how the Baseten platform is designed? How have the design and goals of the project changed or evolved since you started working on it? How do you handle sandboxing of arbitrary user-managed code to ensure security and stability of the platform? How did you approach the system design to allow for mapping application development paradigms into a structure that was accessible to ML professionals? Can you describe the workflow for building an ML powered application? What types of models do you support? (e.g. NLP, computer vision, timeseries, deep neural nets vs. linear regression, etc.) How do the monitoring requirements shift for these different model types? What other challenges are presented by these different model types? What are the limitations in size/complexity/operational requirements that you have to impose to ensure a stable platform? What is the process for deploying model updates? For organizations that are relying on Baseten as a prototyping platform, what are the options for taking a successful application and handing it off to a product team for further customization? What are the most interesting, innovative, or unexpected ways that you have seen Baseten used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Baseten? When is Baseten the wrong choice? What do you have planned for the future of Baseten? Contact Info @tuhinone on Twitter LinkedIn Parting Question From your perspective, what is the biggest barrier to adoption of machine learning today? Closing Announcements Thank you for listening! Don’t forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@themachinelearningpodcast.com) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Links Baseten Gumroad scikit-learn Tensorflow Keras Streamlit Podcast.__init__ Episode Retool Hex Podcast.__init__ Episode Kubernetes React Monaco Huggingface Airtable Dall-E 2 GPT-3 Weights and Biases The intro and outro music is from Hitman’s Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Introducing The Show	03 Jun 2022	00:01:12
Hello, and welcome to the Machine Learning Podcast. I’m your host, Tobias Macey. You might know me from the Data Engineering Podcast or the Python Podcast.__init__. If you work with machine learning and AI, or you’re curious about it and want to learn more, then this show is for you. We’ll go beyond the esoteric research and flashy headlines and find out how machine learning is making an impact on the world and creating value for business. Along the way we’ll be joined by the researchers, engineers, and entrepreneurs who are shaping the industry. So go to themachinelearningpodcast.com today to subscribe and stay informed on how ML/AI are being used, how it works, and how to go from idea to production. Support The Machine Learning Podcast
	The Impact of Generative AI on Software Development	22 Nov 2024	00:52:58
Summary In this episode of the AI Engineering Podcast, Tanner Burson, VP of Engineering at Prismatic, talks about the evolving impact of generative AI on software developers. Tanner shares his insights from engineering leadership and data engineering initiatives, discussing how AI is blurring the lines of developer roles and the strategic value of AI in software development. He explores the current landscape of AI tools, such as GitHub's Copilot, and their influence on productivity and workflow, while also touching on the challenges and opportunities presented by AI in code generation, review, and tooling. Tanner emphasizes the need for human oversight to maintain code quality and security, and offers his thoughts on the future of AI in development, the importance of balancing innovation with practicality, and the evolving role of engineers in an AI-driven landscape. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Your host is Tobias Macey and today I'm interviewing Tanner Burson about the impact of generative AI on software developers Interview Introduction How did you get involved in machine learning? Can you describe what types of roles and work you consider encompassed by the term "developers" for the purpose of this conversation? How does your work at Prismatic give you visibility and insight into the effects of AI on developers and their work? There have been many competing narratives about AI and how much of the software development process it is capable of encompassing. What is your top-level view on what the long-term impact on the job prospects of software developers will be as a result of generative AI? There are many obvious examples of utilities powered by generative AI that are focused on software development. What do you see as the categories or specific tools that are most impactful to the development cycle? In what ways do you find familiarity with/understanding of LLM internals useful when applying them to development processes? As an engineering leader, how are you evaluating and guiding your team on the use of AI powered tools? What are some of the risks that you are guarding against as a result of AI in the development process? What are the most interesting, innovative, or unexpected ways that you have seen AI used in the development process? What are the most interesting, unexpected, or challenging lessons that you have learned while using AI for software development? When is AI the wrong choice for a developer? What are your projections for the near to medium term impact on the developer experience as a result of generative AI? Contact Info LinkedIn Parting Question From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links Prismatic Google AI Development announcement Tabnine Podcast Episode GitHub Copilot Plandex OpenAI API Amazon Q Ollama Huggingface Transformers Anthropic Langchain Llamaindex Haystack Llama 3.2 Qwen2.5-Coder The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	ML Infrastructure Without The Ops: Simplifying The ML Developer Experience With Runhouse	11 Nov 2024	01:16:12
Summary Machine learning workflows have long been complex and difficult to operationalize. They are often characterized by a period of research, resulting in an artifact that gets passed to another engineer or team to prepare for running in production. The MLOps category of tools have tried to build a new set of utilities to reduce that friction, but have instead introduced a new barrier at the team and organizational level. Donny Greenberg took the lessons that he learned on the PyTorch team at Meta and created Runhouse. In this episode he explains how, by reducing the number of opinions in the framework, he has also reduced the complexity of moving from development to production for ML systems. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Your host is Tobias Macey and today I'm interviewing Donny Greenberg about Runhouse and the current state of ML infrastructure Interview Introduction How did you get involved in machine learning? What are the core elements of infrastructure for ML and AI? How has that changed over the past ~5 years? For the past few years the MLOps and data engineering stacks were built and managed separately. How does the current generation of tools and product requirements influence the present and future approach to those domains? There are numerous projects that aim to bridge the complexity gap in running Python and ML code from your laptop up to distributed compute on clouds (e.g. Ray, Metaflow, Dask, Modin, etc.). How do you view the decision process for teams trying to understand which tool(s) to use for managing their ML/AI developer experience? Can you describe what Runhouse is and the story behind it? What are the core problems that you are working to solve? What are the main personas that you are focusing on? (e.g. data scientists, DevOps, data engineers, etc.) How does Runhouse factor into collaboration across skill sets and teams? Can you describe how Runhouse is implemented? How has the focus on developer experience informed the way that you think about the features and interfaces that you include in Runhouse? How do you think about the role of Runhouse in the integration with the AI/ML and data ecosystem? What does the workflow look like for someone building with Runhouse? What is involved in managing the coordination of compute and data locality to reduce networking costs and latencies? What are the most interesting, innovative, or unexpected ways that you have seen Runhouse used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Runhouse? When is Runhouse the wrong choice? What do you have planned for the future of Runhouse? What is your vision for the future of infrastructure and developer experience in ML/AI? Contact Info LinkedIn Parting Question From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links Runhouse GitHub PyTorch Podcast.__init__ Episode Kubernetes Bin Packing Linear Regression Gradient Boosted Decision Tree Deep Learning Transformer Architecture) Slurm Sagemaker Vertex AI Metaflow Podcast.__init__ Episode MLFlow Dask Data Engineering Podcast Episode Ray Podcast.__init__ Episode Spark Databricks Snowflake ArgoCD PyTorch Distributed Horovod Llama.cpp Prefect Data Engineering Podcast Episode Airflow OOM == Out of Memory Weights and Biases KNative BERT language model The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Building AI Systems on Postgres: An Inside Look at pgai Vectorizer	11 Nov 2024	00:53:50
Summary With the growth of vector data as a core element of any AI application comes the need to keep those vectors up to date. When you go beyond prototypes and into production you will need a way to continue experimenting with new embedding models, chunking strategies, etc. You will also need a way to keep the embeddings up to date as your data changes. The team at Timescale created the pgai Vectorizer toolchain to let you manage that work in your Postgres database. In this episode Avthar Sewrathan explains how it works and how you can start using it today. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Your host is Tobias Macey and today I'm interviewing Avthar Sewrathan about the pgai extension for Postgres and how to run your AI workflows in your database Interview Introduction How did you get involved in machine learning? Can you describe what pgai Vectorizer is and the story behind it? What are the benefits of using the database engine to execute AI workflows? What types of operations does pgai Vectorizer enable? What are some common generative AI patterns that can't be done with pgai? AI applications require a large and complex set of dependencies. How does that work with pgai Vectorizer and the Python runtime in Postgres? What are some of the other challenges or system pressures that are introduced by running these AI workflows in the database context? Can you describe how the pgai extension is implemented? With the rapid pace of change in the AI ecosystem, how has that informed the set of features that make sense in pgai Vectorizer and won't require rebuilding in 6 months? Can you describe the workflow of using pgai Vectorizer to build and maintain a set of embeddings in their database? How can pgai Vectorizer help with the situation of migrating to a new embedding model and having to reindex all of the content? How do you think about the developer experience for people who are working with pgai Vectorizer, as compared to using e.g. LangChain, LlamaIndex, etc.? What are the most interesting, innovative, or unexpected ways that you have seen pgai Vectorizer used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on pgai Vectorizer? When is pgai Vectorizer the wrong choice? What do you have planned for the future of pgai Vectorizer? Contact Info LinkedIn Website Parting Question From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links Timescale pgai Transformer architecture for deep learning Neural Networks pgvector pgvectorscale Modal RAG == Retrieval Augmented Generation Semantic Search Ollama GraphRAG agensgraph LangChain LlamaIndex Haystack IVFFlat HNSW DiskANN Repl.it Agent BM25 TSVector ParadeDB The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Running Generative AI Models In Production	28 Oct 2024	00:57:37
Summary In this episode Philip Kiely from BaseTen talks about the intricacies of running open models in production. Philip shares his journey into AI and ML engineering, highlighting the importance of understanding product-level requirements and selecting the right model for deployment. The conversation covers the operational aspects of deploying AI models, including model evaluation, compound AI, and model serving frameworks such as TensorFlow Serving and AWS SageMaker. Philip also discusses the challenges of model quantization, rapid model evolution, and monitoring and observability in AI systems, offering valuable insights into the future trends in AI, including local inference and the competition between open source and proprietary models. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Your host is Tobias Macey and today I'm interviewing Philip Kiely about running open models in production Interview Introduction How did you get involved in machine learning? Can you start by giving an overview of the major decisions to be made when planning the deployment of a generative AI model? How does the model selected in the beginning of the process influence the downstream choices? In terms of application architecture, the major patterns that I've seen are RAG, fine-tuning, multi-agent, or large model. What are the most common methods that you see? (and any that I failed to mention) How have the rapid succession of model generations impacted the ways that teams think about their overall application? (capabilities, features, architecture, etc.) In terms of model serving, I know that Baseten created Truss. What are some of the other notable options that teams are building with? What is the role of the serving framework in the context of the application? There are also a large number of inference engines that have been released. What are the major players in that arena? What are the features and capabilities that they are each basing their competitive advantage on? For someone who is new to AI Engineering, what are some heuristics that you would recommend when choosing an inference engine? Once a model (or set of models) is in production and serving traffic it's necessary to have visibility into how it is performing. What are the key metrics that are necessary to monitor for generative AI systems? In the event that one (or more) metrics are trending negatively, what are the levers that teams can pull to improve them? When running models constructed with e.g. linear regression or deep learning there was a common issue with "concept drift". How does that manifest in the context of large language models, particularly when coupled with performance optimization? What are the most interesting, innovative, or unexpected ways that you have seen teams manage the serving of open gen AI models? What are the most interesting, unexpected, or challenging lessons that you have learned while working with generative AI model serving? When is Baseten the wrong choice? What are the future trends and technology investments that you are focused on in the space of AI model serving? Contact Info LinkedIn Twitter Parting Question From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links Baseten Podcast Episode Copyleft Llama Models Nomic Olmo Allen Institute for AI Playground 2 The Peace Dividend Of The SaaS Wars Vercel Netlify RAG == Retrieval Augmented Generation Podcast Episode Compound AI Langchain Outlines Structured output for AI systems Truss Chains Llamaindex Ray MLFlow Cog (Replicate) containers for ML BentoML Django WSGI uWSGI Gunicorn Zapier vLLM TensorRT-LLM TensorRT Quantization LoRA Low Rank Adaptation of Large Language Models Pruning Distillation Grafana Speculative Decoding Groq Runpod Lambda Labs The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Enhancing AI Retrieval with Knowledge Graphs: A Deep Dive into GraphRAG	10 Sep 2024	00:59:06
Summary In this episode of the AI Engineering podcast, Philip Rathle, CTO of Neo4J, talks about the intersection of knowledge graphs and AI retrieval systems, specifically Retrieval Augmented Generation (RAG). He delves into GraphRAG, a novel approach that combines knowledge graphs with vector-based similarity search to enhance generative AI models. Philip explains how GraphRAG works by integrating a graph database for structured data storage, providing more accurate and explainable AI responses, and addressing limitations of traditional retrieval systems. The conversation covers technical aspects such as data modeling, entity extraction, and ontology use cases, as well as the infrastructure and workflow required to support GraphRAG, setting the stage for innovative applications across various industries. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Your host is Tobias Macey and today I'm interviewing Philip Rathle about the application of knowledge graphs in AI retrieval systems Interview Introduction How did you get involved in machine learning? Can you describe what GraphRAG is? What are the capabilities that graph structures offer beyond vector/similarity-based retrieval methods of prompting? What are some examples of the ways that semantic limitations of nearest-neighbor vector retrieval fail to provide relevant results? What are the technical requirements to implement graph-augmented retrieval? What are the concrete ways in which the embedding and retrieval steps of a typical RAG pipeline need to be modified to account for the addition of the graph? Many tutorials for building vector-based knowledge repositories skip over considerations around data modeling. For building a graph-based knowledge repository there obviously needs to be a bit more work put in. What are the key design choices that need to be made for implementing the graph for an AI application? How does the selection of the ontology/taxonomy impact the performance and capabilities of the resulting application? Building a fully functional knowledge graph can be a significant undertaking on its own. How can LLMs and AI models help with the construction and maintenance of that knowledge repository? What are some of the validation methods that should be brought to bear to ensure that the resulting graph properly represents the knowledge domain that you are trying to model? Vector embedding and retrieval are a core building block for a majority of AI application frameworks. How much support do you see for GraphRAG in the ecosystem? For the case where someone is using a framework that does not explicitly implement GraphRAG techniques, what are some of the implementation strategies that you have seen be most effective for adding that functionality? What are some of the ways that the combination of vector search and knowledge graphs are useful independent of their combination with language models? What are the most interesting, innovative, or unexpected ways that you have seen GraphRAG used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on GraphRAG applications? When is GraphRAG the wrong choice? What are the opportunities for improvement in the design and implementation of graph-based retrieval systems? Contact Info LinkedIn Parting Question From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links Neo4J GraphRAG Manifesto RAG == Retrieval Augmented Generation Podcast Episode VLDB == Very Large DataBases Knowledge Graph Nearest Neighbor Search PageRank Things Not Strings) Google Knowledge Graph Paper pgvector Pinecone Data Engineering Podcast Episode Tables To Labels NLP == Natural Language Processing Ontology LangChain LlamaIndex RLHF == Reinforcement Learning with Human Feedback Senzing NeoConverse Cypher query language GQL query standard AWS Bedrock Vertex AI Sequoia Training Data - Klarna episode Ouroboros The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	GPU Clouds, Aggregators, and the New Economics of AI Compute	27 Jan 2026	00:46:02
Summary In this episode I sit down with Hugo Shi, co-founder and CTO of Saturn Cloud, to map the strategic realities of sourcing and operating GPUs across clouds. Hugo breaks down today’s provider landscape—from hyperscalers to full-service GPU clouds, bare metal/concierge providers, and emerging GPU aggregators—and how to choose among them based on security posture, managed services, and cost. We explore practical layers of capability (compute, orchestration with Kubernetes/Slurm, storage, networking, and managed services), the trade-offs of portability on “Kubernetes-native” stacks, and the persistent challenge of data gravity. We also discuss current supply dynamics, the growing availability of on-demand capacity as newer chips roll out, and how AMD’s ecosystem is maturing as real competition to NVIDIA. Hugo shares patterns for separating training and inference across providers, why traditional ML is far from dead, and how usage varies wildly across domains like biotech. We close with predictions on consolidation, full‑stack experiences from GPU clouds, financial-style GPU marketplaces, and much-needed advances in reliability for long-running GPU jobs. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Unlock the full potential of your AI workloads with a seamless and composable data infrastructure. Bruin is an open source framework that streamlines integration from the command line, allowing you to focus on what matters most - building intelligent systems. Write Python code for your business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. With native support for ML/AI workloads, Bruin empowers data teams to deliver faster, more reliable, and scalable AI solutions. Harness Bruin's connectors for hundreds of platforms, including popular machine learning frameworks like TensorFlow and PyTorch. Build end-to-end AI workflows that integrate seamlessly with your existing tech stack. Join the ranks of forward-thinking organizations that are revolutionizing their data engineering with Bruin. Get started today at aiengineeringpodcast.com/bruin, and for dbt Cloud customers, enjoy a $1,000 credit to migrate to Bruin Cloud. Your host is Tobias Macey and today I'm interviewing Hugo Shi about the strategic realities of sourcing GPUs in the cloud for your training and inference workloads Interview Introduction How did you get involved in machine learning? Can you start by giving a summary of your understanding of the current market for "cloud" GPUs? How would you characterize the customer base for the "neocloud" providers? How is the access to the GPU compute typically mediated? The predominant cloud providers (AWS, GCP, Azure) have gained market share by offering numerous differentiated services and ease-of-use features. What are the types of services that you might expect from a GPU provider? The "cloud-native" ecosystem was developed with the promise of enabling workload portability, but the realities are often more complicated. What are some of the difficulties that teams encounter when trying to adapt their workloads to these different cloud providers? What are the toolchains/frameworks/architectures that you are seeing as most effective at adapting to these different compute environments? One of the major themes in the 2010s that worked against multi-cloud strategies was the idea of "data gravity". What are the strategies that teams are using to mitigate that tax on their workloads? That is a more substantial impact when dealing with training workloads than for inference compute. How are you seeing teams think about the balance of cost savings vs. operational complexity for those different workloads? What are the most interesting, innovative, or unexpected ways that you have seen teams capitalize on GPU capacity across these new providers? What are the most interesting, unexpected, or challenging lessons that you have learned while working on enabling teams to execute workloads on these neoclouds? When is a "neocloud" or "GPU cloud" provider the wrong choice? What are your predictions for the future evolutions of GPU-as-a-service as hardware availability improves and model architectures become more efficient? Contact Info LinkedIn Parting Question From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links Saturn Cloud Pandas NumPy MatLab AWS GCP Azure Oracle Cloud RunPod FluidStack SFCompute KubeFlow Lightning AI DStack Metaflow Flyte Arya AI Dagster Coreweave Vultr Nebius Vast.ai Weka Vast Data Slurm CNCF == Cloud-Native Computing Foundation Kubernetes Terraform ECS Helm Chart Block Storage Object Storage Container Registry Crusoe Alluxio Data Virtualization GB300 H100 Spot Instance AWS Trainium Google TPU (Tensor Processing Unit) AMD ROCM PyTorch Google Vertex AI AWS Bedrock CUDA Python Mojo XGBoost Random Forest Ludwig - Uber Deep Learning AutoML Paperspace Voltage Park Weights & Biases The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	The Future of Dev Experience: Spotify’s Playbook for Organization‑Scale AI	20 Jan 2026	00:56:17
Summary In this episode of the AI Engineering Podcast Niklas Gustavsson, Chief Architect at Spotify, talks about scaling AI across engineering and product. He explores how Spotify's highly distributed architecture was built to support rapid adoption of coding agents like Copilot, Cursor, and Claude Code, enabled by standardization and Backstage. The conversation covers the tension between bottoms-up experimentation and platform standardization, and how Spotify is moving toward monorepos and fleet management. Niklas discusses the emergence of "fleet-wide agents" that can execute complex code changes with robust testing and LLM-as-judge loops to ensure quality. He also touches on the shift in engineering workflows as code generation accelerates, the growing use of agents beyond coding, and the lessons learned in sandboxing, agent skills/rules, and shared evaluation frameworks. Niklas highlights Spotify's decade-long experience with ML product work and shares his vision for deeper end-to-end integration of agentic capabilities across the full product lifecycle and making collaborative "team-level memory" for agents a reality. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems Unlock the full potential of your AI workloads with a seamless and composable data infrastructure. Bruin is an open source framework that streamlines integration from the command line, allowing you to focus on what matters most - building intelligent systems. Write Python code for your business logic, and let Bruin handle the heavy lifting of data movement, lineage tracking, data quality monitoring, and governance enforcement. With native support for ML/AI workloads, Bruin empowers data teams to deliver faster, more reliable, and scalable AI solutions. Harness Bruin's connectors for hundreds of platforms, including popular machine learning frameworks like TensorFlow and PyTorch. Build end-to-end AI workflows that integrate seamlessly with your existing tech stack. Join the ranks of forward-thinking organizations that are revolutionizing their data engineering with Bruin. Get started today at aiengineeringpodcast.com/bruin, and for dbt Cloud customers, enjoy a $1,000 credit to migrate to Bruin Cloud. Your host is Tobias Macey and today I'm interviewing Niklas Gustavsson about how Spotify is scaling AI usage in engineering and product work Interview Introduction How did you get involved in machine learning? Can you start by giving an overview of your engineering practices independent of AI? What was your process for introducing AI into the developmer experience? (e.g. pioneers doing early work (bottom-up) vs. top-down) There are countless agentic coding tools on the market now. How do you balance organizational standardization vs. exploration? Beyond the toolchain, what are your methods for sharing best practices and upskilling engineers on use of agentic toolchains for software/product engineering? Spotify has been operationalizing ML/AI features since before the introduction of LLMs and transformer models. How has that history helped inform your adoption of generative AI in your overall engineering organization? As you use these generative and agentic AI utilities in your day-to-day, how have those lessons learned fed back into your AI-powered product features? What are some of the platform capabilities/developer experience investments that you have made to improve the overall effectiveness of agentic coding in your engineering organization? What are some examples of guardrails/speedbumps that you have introduced to avoid injecting unreliable or untested work into production? As the (time/money/cognitive) cost of writing code drops that increases the burden on reviewing that code. What are some of the ways that you are working to scale that side of the equation? What are some of the ways that agentic coding/CLI utilities have bled into other areas of engineering/opertions/product development beyond just writing code? What are the most interesting, innovative, or unexpected ways that you have seen your team applying AI/agentic engineering practices? What are the most interesting, unexpected, or challenging lessons that you have learned while working on operationalizing and scaling agentic engineering patterns in your teams? When is agentic code generation the wrong choice? What do you have planned for the future of AI and agentic coding patterns and practices in your organization? Contact Info LinkedIn Parting Question From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links Spotify Developer Experience LLM == Large Language Model Transformers BackStage GitHub Copilot Cursor Claude Skills Monorepo MCP == Model Context Protocol Claude Code Product Manager DORA Metrics Type Annotations BigQuery PRD == Product Requirements Document AI Evals LLM-as-a-Judge Agentic Memory The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0
	Specs, Tests, and Self‑Verification: The Playbook for Agentic Engineering Teams	19 Oct 2025	01:06:28
Summary In this episode Andrew Filev, CEO and founder of ZenCoder, takes a deep dive into the system design, workflows, and organizational changes behind building agentic coding systems. He traces the evolution from autocomplete to truly agentic models, discusses why context engineering and verification are the real unlocks for reliability, and outlines a pragmatic path from “vibe coding” to AI‑first engineering. Andrew shares ZenCoder’s internal playbook: PRD and tech spec co‑creation with AI, human‑in‑the‑loop gates, test‑driven development, and emerging BDD-style acceptance testing. He explores multi-repo context, cross-service reasoning, and how AI reshapes team communication, ownership, and architecture decisions. He also covers cost strategies, when to choose agents vs. manual edits, and why self‑verification and collaborative agent UX will define the next wave. Andrew offers candid lessons from building ZenCoder—why speed of iteration beats optimizing for weak models, how ignoring the emotional impact of vibe coding slowed brand momentum, and where agentic tools fit across greenfield and legacy systems. He closes with predictions for the next year: self‑verification, parallelized agent workflows, background execution in CI, and collaborative spec‑driven development moving code review upstream. Announcements Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems When ML teams try to run complex workflows through traditional orchestration tools, they hit walls. Cash App discovered this with their fraud detection models - they needed flexible compute, isolated environments, and seamless data exchange between workflows, but their existing tools couldn't deliver. That's why Cash App rely on Prefect. Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS, and Databricks. Custom packages stay isolated. Model outputs flow seamlessly between workflows. Companies like Whoop and 1Password also trust Prefect for their critical workflows. But Prefect didn't stop there. They just launched FastMCP - production-ready infrastructure for AI tools. You get Prefect's orchestration plus instant OAuth, serverless scaling, and blazing-fast Python execution. Deploy your AI tools once, connect to Claude, Cursor, or any MCP client. No more building auth flows or managing servers. Prefect orchestrates your ML pipeline. FastMCP handles your AI tool infrastructure. See what Prefect and Fast MCP can do for your AI workflows at aiengineeringpodcast.com/prefect today. Your host is Tobias Macey and today I'm interviewing Andrew Filev about the system design and integration strategies behind building coding agents at Zencoder Interview Introduction How did you get involved in ML/AI? There have been several iterations of applications for generative AI models in the context of software engineering. How would you characterize the different approaches or categories? Over the course of this summer (2025) the term "vibe coding" gained prominence with the idea that the human just needs to be worried about whether the software does what you ask, not how it is written. How does that sentiment compare to your philosophies on the role of agentic AI in the lifecycle of software? This points at a broader challenge for software engineers in the AI era; how much control can and should we cede to the LLMs, and over what elements of the software process? This also brings up useful questions around the experience of the engineer collaborating with the agent. What are the different interaction patterns that individuals and teams should be thinking of in their use of AI engineering tools? Should the agent be proactive? reactive? what are the triggers for an action to be taken and to what extent? What differentiates a coding agent from an agentic editor? The key challenge in any agent system is context engineering. Software is inherently structured and provides strong feedback loops. But it can also be very messy or difficult to encapsulate in a single context window. What are some of the data structures/indexing strategies/retrieval methods that are most useful when providing guidance to an agent? Software projects are rarely fully self-contained, and often need to cross repository boundaries, as well as manage dependencies. What are some of the more challenging aspects of identifying and accounting for those sometimes implicit relationships? What are some of the strategies that are most effective for yielding productive results from an agent in terms of prompting and scoping of the problem? What are some of the heuristics that you use to determine whether and how to employ an agent for a given task vs. doing it manually? How can the agents assist in the decomposition and planning of complex projects? What are some of the ways that single-player interaction strategies can be turned into team/multi-player strategies? What are some of the ways that teams can create and curate productive patterns to accelerate everyone equally? What are the most interesting, innovative, or unexpected ways that you have seen coding agents used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on coding agents at Zencoder? When is/are Zencoder/coding agents the wrong choice? What do you have planned for the future of Zencoder/agentic software engineering? Contact Info LinkedIn Parting Question From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers. Links Zencoder Wrike DARPA Robotics Challenge Cognitive Computing Andrew Ng Sebastian Thrun Github Copilot RAG == Retrieval Augmented Generation Re-ranking Claude Sonnet 3.5 SWE-Bench Vibe Coding AI First Engineering Waterfall Software Engineering Agile Software Engineering PRD == Project Requirements Document BDD == Behavior-Driven Development VSCode The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0

About us Privacy Policy