The AI Dream Team: Strategies for ML Recruitment and Growth // MLOps Podcast #267 with Jelmer Borst, Analytics & Machine Learning Domain Lead, and Daniela Solis, Machine Learning Product Owner, of Picnic.

// Abstract

Like many companies, Picnic started out with a small, central data science team. As this grows larger, focusing on more complex models, it questions the skillsets & organisational setup. Use an ML platform, or build ourselves? A central team vs. embedded? Hire data scientists vs. ML engineers vs. MLOps engineers. How to foster a team culture of end-to-end ownership to balance short-term & long-term impact

// Bio

Jelmer Borst

Jelmer leads the analytics & machine learning teams at Picnic, an app-only online groceries company based in the Netherlands. Whilst his background is in aerospace engineering, he was looking for something faster-paced and found that at Picnic. He loves the intersection of solving business challenges using technology & data. In his free time loves to cook food and tinker with the latest AI developments.

Daniela Solis Morales

As a Machine Learning Lead at Picnic, I am responsible for ensuring the success of end-to-end Machine Learning systems. My work involves bringing models into production across various domains, including Personalization, Fraud Detection, and Natural Language Processing.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Jelmer on LinkedIn: https://www.linkedin.com/in/japborst

Connect with Daniela on LinkedIn: https://www.linkedin.com/in/daniela-solis-morales/

Timestamps:

[00:00] Jelmer and Daniela's preferred coffee

[00:37] Takeaways

[03:46] Please like, share, leave a review, and subscribe to our MLOps channels!

[03:58] Use case evolution review

[08:24] Centralized ML strategy

[11:53] Managing zombie models effectively

[15:52] Clean data and collaboration

[21:07] Snowflake ML Integration options

[22:49] MLOps infrastructure components

[25:36] Pull vs. Push Adoption

[27:03] ML Model Monitoring Roles

[31:56] Inventory prediction

[36:00] Scaling machine learning teams

[42:09] Team expansion and structure

[48:20] Exploring effective team organization

[51:43] Blog reading insights

[54:25] Playing hard mode

[57:33] Wrap up

Francisco Ingham, LLM consultant, NLP developer, and founder of Pampa Labs.Making Your Company LLM-native

// MLOps Podcast #266 with Francisco Ingham, Founder of Pampa Labs.

// Abstract

Being an LLM-native is becoming one of the key differentiators among companies in vastly different verticals. Everyone wants to use LLMs, and everyone wants to be on top of the current tech, but what does it really mean to be LLM-native?

LLM-native involves two ends of a spectrum. On the one hand, we have the product or service that the company offers, which surely offers many automation opportunities. LLMs can be applied strategically to scale at a lower cost and offer a better experience for users.

But being LLM-native not only involves the company's customers, it also involves each stakeholder involved in the company's operations. How can employees integrate LLMs into their daily workflows? How can we, as developers, leverage the advancements in the field not only as builders but as adopters?

We will tackle these and other key questions for anyone looking to capitalize on the LLM wave, prioritizing real results over the hype.

// Bio

Currently working at Pampa Labs, where we help companies become AI-native and build AI-native products. Our expertise lies on the LLM-science side, or how to build a successful data flywheel to leverage user interactions to continuously improve the product. We also spearhead Pampa-friends - the first Spanish-speaking community of AI Engineers.

Previously worked in management consulting, was a TA in fastai in SF, and led the cross-AI + dev tools team at Mercado Libre.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: pampa.ai

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Francisco on LinkedIn: https://www.linkedin.com/in/fpingham/

Timestamps:

[00:00] Francisco's preferred coffee

[00:13] Takeaways

[00:37] Please like, share, leave a review, and subscribe to our MLOps channels!

[00:51] A Literature Geek

[02:41] LLM-native company

[03:54] Integrating LLM in workflows

[07:21] Unexpected LLM applications

[10:38] LLMs in the development process

[14:00] Vibe check to evaluation

[15:36] Experiment tracking optimizations

[20:22] LLMs as judges discussion

[24:43] Presentaciones automatizadas para podcast

[27:48] AI operating system and agents

[31:29] Importance of SEO expertise

[35:33] Experimentation and evaluation

[39:20] AI integration strategies

[41:50] RAG approach spectrum analysis

[44:40] Search vs Retrieval in AI

[49:02] Recommender Systems vs RAG

[52:08] LLMs in recommender systems

[53:10] LLM interface design insights

Markus Stoll is the Co-Founder of Renumics and the developer behind the open-source interactive ML dataset exploration tool, Spotlight. He shares insights on:

AI in Engineering and Manufacturing
Interactive ML Data Visualization
ML Data Exploration

Follow Markus for hands-on articles about leveraging ML while keeping a strong focus on data.

Visualize - Bringing Structure to Unstructured Data // MLOps Podcast #258 with Markus Stoll, CTO of Renumics.

A huge thank you to SAS for their generous support!

// Abstract

This talk is about how data visualization and embeddings can support you in understanding your machine-learning data. We explore methods to structure and visualize unstructured data like text, images, and audio for applications ranging from classification and detection to Retrieval-Augmented Generation. By using tools and techniques like UMAP to reduce data dimensions and visualization tools like Renumics Spotlight, we aim to make data analysis for ML easier. Whether you're dealing with interpretable features, metadata, or embeddings, we'll show you how to use them all together to uncover hidden patterns in multimodal data, evaluate the model performance for data subgroups, and find failure modes of your ML models.

// Bio

Markus Stoll began his career in the industry at Siemens Healthineers, developing software for the Heavy Ion Therapy Center in Heidelberg. He learned about software quality while developing a treatment machine weighing over 600 tons. He earned a Ph.D., focusing on combining biomechanical models with statistical models, through which he learned how challenging it is to bridge the gap between research and practical application in the healthcare domain. Since co-founding Renumics, he has been active in the field of AI for Engineering, e.g., AI for Computer Aided Engineering (CAE), implementing projects, contributing to their open-source library for data exploration for ML datasets (Renumics Spotlight), and writing articles about data visualization.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://renumics.com/

MLSecOps Community: https://community.mlsecops.com/

Blogs: https://towardsdatascience.com/visualize-your-rag-data-evaluate-your-retrieval-augmented-generation-system-with-ragas-fc2486308557 : https://medium.com/itnext/how-to-explore-and-visualize-ml-data-for-object-detection-in-images-88e074f46361

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Markus on LinkedIn: https://www.linkedin.com/in/markus-stoll-b39a42138/

Timestamps:

[00:00] Markus' preferred coffee

[00:15] Takeaways

[01:41] Please like, share, leave a review, and subscribe to our MLOps channels!

[01:50] Register for the Data Engineering for AI/ML Conference now!

[02:27] Current focus and updates

[04:43] 3D Embeddings Visualization Explained

[07:07] Question Embeddings vs Retrieval

[08:24] Using heat maps effectively

[10:30] User insights visualization RAG

[16:59] 3D Crash Simulation Analysis

[20:33] Simulation purpose clarification

[22:34] Evaluating test data use cases

[24:22] Real-world car testing

[29:48] Identifying data issues early

[33:33] Multimodal data integration

[37:42] Custom vs Fine-tuned models

[39:45] Data processing challenges

[45:58] Use case-driven MVP

[48:26 - 50:08] SAS Ad

[50:09] Wrap up

MLOps podcast #186 with Mike Del Balso, CEO & Co-founder of Tecton and Josh Wills, Angel Investor, The Future of Feature Stores and Platforms.

// Abstract

Mike and Josh discuss creating templates and working at a detailed level, exploring Tecton's potential for sharing fraud and third-party features. They focus on technical aspects like data handling and optimizing models, emphasizing the significance of quality data for AI systems and the necessity for cohesive feature infrastructure in reaching production stages.

// Bio

Mike Del Balso

Mike is the co-founder of Tecton, where he is focused on building next-generation data infrastructure for Operational ML. Before Tecton, Mike was the PM lead for the Uber Michelangelo ML platform. He was also a product manager at Google, where he managed the core ML systems that power Google’s Search Ads business.

Josh Wills

Josh Wills is an angel investor specializing in data and machine learning infrastructure. He was formerly the head of data engineering at Slack, the director of data science at Cloudera, and a software engineer at Google.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links⁠

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Mike on LinkedIn: https://www.linkedin.com/in/michaeldelbalso/

Connect with Josh on LinkedIn: https://www.linkedin.com/in/josh-wills-13882b/

Timestamps:

[00:00] Introduction to Mike

[01:45] Takeaways

[03:32] Features of the new paradigm of ML and LLMs

[06:00] D. Sculley's papers

[13:05] The birth of Feature Store

[17:06] Data Pipeline Challenges Addressed

[20:00] Operationalizing

[26:50] Feature Store Challenges

[30:26] Z access

[36:23] Addressing Technical Debt Challenges

[37:27] Real-Time vs. Batch Processing

[47:10] Feature Store Evolution: Apache Iceberg

[49:59] Feature Platform: Dedicated Query Engine

[54:04] The bottleneck

[56:00] LLMs, Feature Stores Overview

[1:00:20] Vector databases

[1:06:15] Workflow Templating Efficiency

[1:08:35] Gamification suggestion for Tecton

[1:10:25] Wrap up

MLOps podcast #185 with Luigi Patruno, VP of Data Science at 2U, Inc., Lessons on Data Science Leadership.

// Abstract

Picture this: you've got data products to manage, and you're in charge of a team. It's not all sunshine and rainbows, right? Luigi dives into the nitty-gritty of the challenges - from juggling data projects to wrangling the team dynamics. It's a real adventure, let me tell you!

// Bio

Luigi Patruno is a results-driven data science leader passionate about identifying value-add business opportunities and converting these into analytical solutions that deliver measurable business outcomes. As a leader, he focuses on defining strategic vision and, through motivation and discipline, driving teams of highly quantitative data scientists, machine learning engineers, and product managers to achieve extraordinary results. He is currently the VP of Data Science at 2U, where he leads the data science department focused on optimizing business operations through advanced analytics, experimentation, and machine learning. He enjoys teaching others how to leverage data science to improve their businesses through public speaking, teaching courses, and writing online at MLinProduction.com.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://mlinproduction.com/

YouTube channel: https://www.youtube.com/playlist?list=PLBLnN4jzkyqkjLIRpDNZcsG7TMMEk9Asa

High Output Management book by Andrew Grove: https://www.amazon.nl/-/en/Andrew-S-Grove/dp/0679762884

The One Minute Manager by Kenneth Blanchard, Ph.D., and Spencer Johnson, M.D.: https://www.amazon.com/Minute-Manager-Kenneth-Blanchard-Ph-D/dp/074350917X

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Luigi on LinkedIn: https://www.linkedin.com/in/luigipatruno91/

Timestamps:

[00:00] Luigi's preferred coffee

[00:30] Takeaways

[03:04] Being practical

[05:44] Data-Driven Decision-Making in Management

[12:53] Recent Team Win

[14:43] The perfect storm

[20:22] Change Management and ROI

[25:09] Change Management: Navigating Resistance

[29:59] Clarifying North Star Communication

[36:24] OKRs in Data Science

[40:47] Success Likelihood in Business

[45:08] Bus problem solution

[49:25] Data Science-Platform Collaboration

[53:19] Decentralized Platforms Explained

[54:38] Data Platform Architecture Overview

[57:14] Incentives for Team Motivation

[1:09:45] The blind spots

[1:12:22] Wrap up

MLOps podcast #184 with Richa Sachdev, Executive Director- Data Operations and Automation at JP Morgan Chase, Data Platforms in MLOps: Translating Business Goals into Product Decisions.

// Abstract

Richa, with her background in software engineering and experience in the financial sector, shares her insights on optimizing the end-user experience and the importance of understanding business goals and metrics. She discusses her journey in converting legacy applications, working with data platforms, and the challenges of integrating different databases. Richa also explores the role of automation in streamlining processes and improving customer interactions in the reward space. Join us as we unravel the fascinating world of MLOps and uncover the strategies and technologies that drive success in this ever-evolving field.

// Bio

A passionate and impact-driven leader whose expertise spans leading teams, architecting ML and data-intensive applications, and driving enterprise data strategy. Richa has worked for a Tier A Start-up developing feature platforms and in financial companies, leading ML Engineering teams to drive data-driven business decisions. Richa enjoys reading technical blogs focused on system design and plays an active role in the MLOps Community.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://www.youtube.com/watch?v=i0To3DeHGuU
https://www.youtube.com/watch?v=tAOf2lVQUY4
https://www.youtube.com/watch?v=cXanVyaannQ
https://www.youtube.com/watch?v=2aWSsL24fv8

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Richa on LinkedIn: https://www.linkedin.com/in/richasachdev/

Timestamps:

[00:00] Richa's preferred coffee

[02:09] Takeaways

[04:26] Richa's background in data

[08:55] Prescriptive, Descriptive, and Predictive Data

[11:50] Data Engineering Perspectives & Setup

[17:34] Structured and Unstructured data

[21:01] Richa's day-to-day at Chase

[23:52] Figure out the business needs before the cool tech

[26:46] Importance of business metrics

[30:43] Optimizing end-user experience and trade-offs

[36:06] Exhausting creativity in finding solutions

[37:40] Consider faster implementation and increased ROI

[40:20] Banks still using COBOL

[41:17] Learning and growing as a versatile leader

[42:04] Wrap up

MLOps podcast #183 with Ketan Umare, CEO of Union.AI, MLOps vs ML Orchestration, co-hosted by Stephen Batifol.

// Abstract

Let's explore the relationship between Union and Flyte, emphasizing the significance of community-driven development and the challenge of balancing feature requests with security considerations. This conversation highlights the importance of real-time data and secure data handling in orchestrating machine learning models. The Flyte community's empathy and support for newcomers underscore the community's value in democratizing machine learning, making it more accessible and efficient for a broader audience.

// Bio

Ketan Umare is the CEO and co-founder at Union.ai. Previously, he had multiple Senior roles at Lyft, Oracle, and Amazon ranging from Cloud, distributed storage, Mapping (map-making), and machine-learning systems. He is passionate about building software that makes engineers' lives easier and provides simplified access to large-scale systems. Besides software, he is a proud father and husband, and enjoys traveling and outdoor activities.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://union.ai/

Flyte: https://flyte.org/ ⁠

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn:

https://www.linkedin.com/in/dpbrinkm/

Connect with Stephen on LinkedIn: https://www.linkedin.com/in/stephen-batifol/

Connect with Ketan on LinkedIn: https://www.linkedin.com/in/ketanumare/

Timestamps:

[00:00] Ketan's preferred coffee

[01:05] Takeaways

[03:08] Please like, share, and subscribe to our MLOps channels!

[03:15] Shout out to Ketan and UnionAI for sponsoring this episode!

[04:23] Orchestration of recent changes

[07:51] Community with Flyte

[11:26] ML orchestration

[15:40] 50/50 is generous

[20:06] Real-time ML

[21:15] Over-engineering without benefits

[23:20] Balancing everything

[27:40] Union verse Flyte

[32:52] High-value features of Union AI at the back of Flyte

[40:18] Building LLM infrastructure

[45:30] Traditional ML is the whole prompting

[46:46] LLMs for evaluating prompts

[48:55] Wrap up

MLOps podcast #182 with GetYourGuide's Jean Machado, DataScience Manager, Meghana Satish, MLOps Engineer, Olivia Houghton, Machine Learning Operations Engineer, Theodore Meynard, Data Science Manager, MLOps@GetYourGuide.

// Abstract

Join a team to talk about the journey of GYG with MLOps, from the conception of their platform to the creation of the MLOps engineer role, and to their current stack state.

// Bio

Jean Machado

Jean Carlo Machado is a Data Science Manager at GetYourGuide for the Growth Data Products team and the Machine Learning Platform Team. He is privileged to be able to work on turning ideas in data science from inception to production. Before GYG, Jean was working in a startup in Brazil, building its infrastructure from the ground up. Jean also likes community building and using technology for social good.

Meghana Satish

Meghana Satish is currently working as an MLOps Engineer at GetYourGuide. She has previously held positions at Amazon AWS in Berlin and Microsoft IT in Hyderabad. In addition to her career in technology, Meghana is also a talented singer, dancer, and yoga practitioner.

Olivia Houghton

Olivia has been working as an MLOps engineer at GetYourGuide for the past year and a half or so. Olivia's main work is in building and managing their activity ranking service.

Theodore Meynard

Theodore Meynard, Data Science Manager at GetYourGuide, leads the evolution of their ranking algorithm, enriching customer experiences. His hands-on journey from data scientist to leader has honed his expertise in MLOps and real-time ML. Beyond work, he's a co-organizer of PyData Berlin, underlining his commitment to community and collaborative learning.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://investing1012dot0.substack.com/

The Openness of AI report: https://research.contrary.com/reports/the-openness-of-ai

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Jean on LinkedIn: https://www.linkedin.com/in/jean-carlo-machado-53b15977/

Connect with Meghana on LinkedIn: https://www.linkedin.com/in/meghana-satish-2a825282/?originalSubdomain=de

Connect with Olivia on LinkedIn: https://www.linkedin.com/in/oliviaphoughton/

Connect with Theodore on LinkedIn: https://www.linkedin.com/in/theodore-meynard/

Timestamps:

[00:00] GetYourGuide team's preferred coffee

[00:55] Takeaways

[02:20] Shout out to Berlin MLOps Community

[02:38] Please like, share, and subscribe to our MLOps channels!

[03:39] The GetYourGuide platform

[05:45] GetYourGuide use cases

[11:51] Strong Leadership Vision

[13:59] Creating rituals

[16:55] Feedback on the loop for improvements

[18:35] Different components of GetYourGuide's ML Platform

[21:04] V2 service templates

[24:26] Biggest pain points

[27:02] Feature flags

[30:51] Data foundation

[36:25] Data Testing

[39:53] Cross-team Tool Adoption Process

[44:59] Regrets about design decisions made in the past

[47:53] What's next for the platform with LLMs?

[52:49] Non-data scientists suggesting use cases, language flexibility

[55:14] DevSecOps team's AI study group ideation

[59:25] Experiments in growth data products, marketing split

[1:01:47] Shout out to the Berlin MLOps Community!

[1:03:31] Wrap up

MLOps podcast #181 with Kyle Harrison, General Partner at Contrary, The Centralization of Power in AI.

// Abstract

Kyle Harrison delves into the limitations imposed by language, underscoring how it can impede our grasp and manipulation of reality while stressing the critical need for improved language model performance for real-time applications. He further explores the perils of centralizing power in AI, with a specific focus on the "Openness of AI", where concerns about privacy are brought to the forefront, prompting his call for businesses to reconsider their reliance on it. The discussion also traverses the evolving landscape of AI, drawing comparisons between prominent machine learning frameworks such as TensorFlow and PyTorch. Notably, the episode underscores the vital role of open-source initiatives within the AI community and highlights the unexpected involvement of Meta in driving open-source development.

// Bio

Kyle Harrison is a General Partner at Contrary, where he leads Series A and growth-stage investing. He joined Contrary from Index, where he was a Partner, and before that, he was a growth investor at Coatue. His portfolio includes iconic startups and public companies, including Ramp, Replit, Cohere, Snowflake, and Databricks. He also regularly shares his analysis on the venture capital landscape via his Substack Investing 101.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related LinksWebsite: https://investing1012dot0.substack.com/

The Openness of AI report: https://research.contrary.com/reports/the-openness-of-ai⁠

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Kyle on LinkedIn: https://www.linkedin.com/in/kyle-harrison-9274b278/

Timestamps:

[00:00] Kyle's preferred beverage

[00:20] Takeaways

[03:52] Hype in the technology space

[09:20] Application Layer Revenue

[14:44] Stability AI Lawsuit

[18:08] Concern over concentration of power in AI

[20:20] Transparency concerns

[23:35] Open Source AI

[25:57] To use or not to use OpenAI

[30:51] Lack of technical expertise and business-building capabilities

[35:09] AI Transparency and Accountability

[37:50] Traditional ML

[41:47] Finding a unique approach

[45:41] AGI limitations

[47:43] Using Agents

[49:46] Agents getting past demos

[54:39] Tech Challenges & Hoverboard Dreams

[58:04] Both AI hype and skepticism are foolish

[01:27] Wrap up

MLOps podcast #180 with Sachin Abeywardana, Deep Learning Engineer at Canva AI, Adventures in Building CLIP and Other (Largeish) Language Models sponsored by Prem AI.

// Abstract

Sachin takes us on an adventure, sharing insights on the pitfalls of not understanding the broader product and the importance of incorporating AI and machine learning capabilities. From the use of AI models for grammar correction and code generation to the fascinating Clip model and the challenges of balancing work and family life, this episode promises to be both informative and thought-provoking.

// Bio

Sachin is the father of two beautiful children. He completed his PhD in Bayesian Machine Learning at the University of Sydney in 2015. In 2016, he discovered Deep Learning and hasn't looked back. He currently works as a Senior Machine Learning Engineer at Canva and is mainly focusing on NLP problems.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Sachin Blogs: https://sachinruk.github.io/blog.htmlhttps://sachinruk.github.io/blog/

Graph ML link: http://web.stanford.edu/class/cs224w/

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Sachin on LinkedIn: https://www.linkedin.com/in/sachinabeywardana/

Timestamps:

[00:00] Sachin's preferred beverage

[00:26] Takeaways

[02:30] Chat GPT user

[05:58] Understanding of reliable Agents

[08:10] Sachin's background

[12:45] Staying at Deep Learning

[16:17] Recommendation or Lead Scoring

[17:36] Vector database

[19:00] Sachin's blogs

[23:26] The cap people

[26:10] Pursuing a business case

[27:33] Canva

[31:16] Incorporating AI and Machine Learning

[32:17] Sponsor Ad

[38:22] Eliminating unnecessary steps

[39:00] Interacting with the product team

[43:04] Criticisms of the current architecture limitations

[45:58] Insufficient exploration of Transformers

[47:42] Explaining GraphML

[52:35] Fine-tuning ChatGPT2

[57:54] Leading ML Engineers and teams

[59:40] Being practical with Math

[1:05:52] Wrap up

MLOps Coffee Sessions #179 with Shahul Es, All About Evaluating LLM Applications.

// Abstract

Shahul Es, renowned for his expertise in the evaluation space and is the creator of the Ragas Project. Shahul dives deep into the world of evaluation in open source models, sharing insights on debugging, troubleshooting, and the challenges faced when it comes to benchmarks. From the importance of custom data distributions to the role of fine-tuning in enhancing model performance, this episode is packed with valuable information for anyone interested in language models and AI.

// Bio

Shahul is a data science professional with 6+ years of expertise and has worked in data domains from structured, NLP, to Audio processing. He is also a Kaggle GrandMaster and code owner/ ML of the Open-Assistant initiative that released some of the best open-source alternatives to ChatGPT.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

All about evaluating Large language models blog: https://explodinggradients.com/all-about-evaluating-large-language-models

Ragas: https://github.com/explodinggradients/ragas

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Shahul on LinkedIn: https://www.linkedin.com/in/shahules/

Timestamps:

[00:00] Shahul's preferred coffee

[00:20] Takeaways

[01:46] Please like, share, and subscribe to our MLOps channels!

[02:07] Shahul's definition of Evaluation

[03:27] Evaluation metrics and Benchmarks

[05:46] Gamed leaderboards

[10:13] Best at summarizing long text open-source models

[11:12] Benchmarks

[14:20] Recommending the evaluation process

[17:43] LLMs for other LLMs

[20:40] Debugging failed evaluation models

[24:25] Prompt injection

[27:32] Alignment

[32:45] Open Assist

[35:51] Garbage in, garbage out

[37:00] Ragas

[42:52] Valuable use case besides OpenAI

[45:11] Fine-tuning LLMs

[49:07] Connect with Shahul if you need help with Ragas @Shahules786 on Twitter

[49:58] Wrap up

MLOps Coffee Sessions #178 with Stephen Batifol, Building an ML Platform: Insights, Community, and Advocacy.

// Abstract

Discover how Wolt onboards data scientists onto the platform and builds a thriving internal community of users. Stephen's firsthand experiences shed light on the importance of developer relations and how they contribute to making data scientists' lives easier. From top-notch documentation to getting-started guides and tutorials, the internal platform at Wolt prioritizes the needs of its users.

// Bio

From Android developer to Data Scientist to Machine Learning Engineer, Stephen has a wealth of software engineering experience at Wolt. He believes that machine learning has a lot to learn from software engineering best practices and spends his time making ML deployments simple for other engineers. Stephen is also a founding member and organizer of the MLOps.community Meetups in Berlin.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Stephen on LinkedIn: https://www.linkedin.com/in/stephen-batifol/

Timestamps:

[00:00] Stephen's preferred coffee

[00:32] Takeaways

[01:35] Please like, share, and subscribe to our MLOps channels!

[03:00] Creating his own team!

[04:44] DevRel

[06:32] The door dash of Europe

[11:28] Data platform underneath

[12:55] Cellular core deployment uses open source

[14:21] Alibi

[16:08] Kafka

[16:59] Selling points to data scientists

[20:05] Language models concern data scientists

[22:12] Incorporating LLMs into the business

[23:55] Feedback from data scientists and end users

[27:37] User surveys

[30:11] Evangelizing and giving talks

[35:25] Tech Hub Culture in Berlin

[38:38] Kubernetes lifestyle

[42:55] Interacting with SREs

[45:28] Wrap up

MLOps Coffee Sessions #176 with Vin Vashishta, Collaboration and Strategy.

// Abstract

From the significance of technical strategists to the crucial role of product managers with a deep understanding of data and AI products, Vin shares invaluable insights on fostering collaboration, driving strategy, and maximizing the potential of data within organizations. Join us as we explore the importance of becoming multipliers in our fields, the power of effective strategy in leveraging data, and the opportunities that lie in the generative AI era.

// Bio

Vin's background is in applied data science. He is the founder of V Squared, one of the oldest and smallest data science consulting companies in the world. They help businesses monetize data and AI. Vin is the author of From Data to Profit. He teaches technical strategy and data, and AI product management.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://www.datascience.vin/

From Data to Profit: How Businesses Leverage Data to Grow Their Top and Bottom Lines book:https://www.amazon.com/Data-Profit-Businesses-Leverage-Bottom/dp/1394196210/⁠

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Vin on LinkedIn: https://www.linkedin.com/in/vineetvashishta/

Timestamps:

[00:00] Vin's preferred coffee

[00:14] Takeaways

[02:02] Please like, share, and subscribe to our MLOps channels!

[02:28] Recent ideas of Vin

[05:09] Understanding the business value of any project [08:30] Generative AI making things faster

[14:29] Strategy in practice

[20:19] Practicality and Credibility of Strategists

[22:42] Coming soon!!! LLMs in Production Conference Panel Part III

[27:48] Becoming a Multiplier

[29:03] The AI Product Manager

[35:12] Successful monetization and integration of technologies

[37:48] Justifying the ROI of LLMs

[44:59] Adding that extra value

[49:52] Read Vin's book linked above!

[50:35] Wrap up

MLOps for GenAI Applications // Special MLOps Podcast episode with Demetrios Brinkmann, Chief Happiness Engineer at MLOps Community.

// Abstract

Demetrios explores common themes in ML model testing with insights from Erica Greene (Yahoo News), Matar Haller (ActiveFence), Mohamed Elgendy (Kolena), and Catherine Nelson (Freelance Data Scientist). They discuss tiered test cases, functional testing for hate speech, differences between AI and traditional software testing, and the complexities of evaluating LLMs. Demetrios wraps up by inviting feedback and promoting an upcoming virtual conference on data engineering for AI and ML.

// Bio

At the moment, Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps Community Podcasts. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Balancing Speed and Safety // Panel // AIQCON - https://youtu.be/c81puRgu3Kw

AI For Good - Detecting Harmful Content at Scale // Matar Haller // MLOps Podcast #246 - https://youtu.be/wLKlZ6yHg1k

What is AI Quality? // Mohamed Elgendy // MLOps Podcast #229 - https://youtu.be/-Jdmq4DiOew

All Data Scientists Should Learn Software Engineering Principles // Catherine Nelson // Podcast #245 - https://youtu.be/yP6Eyny7p20

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Timestamps:

[00:00] Exploring common themes in MLOps community

[00:49] Common patterns about model output and testing

[01:34] Tiered test case strategy

[03:05] Functional testing for models

[05:24] Testing coverage and quality

[07:47] Evaluating LLMs challenges

[08:35] Please like, share, leave a review, and subscribe to our MLOps channels!

Sign up for our next LLM in production conference: https://go.mlops.community/prodiii

#180 with LLMs in Production Conference part 2 Ux of a LLM User Panel, Misty Free, Dina Yerlan, and Artem Harutyunyan hosted by Innovation Endeavors' Davis Treybig. // Abstract Explore different approaches to interface design, emphasizing the significance of crafting effective prompts and addressing accuracy and hallucination issues. Discover some strategies for improving latency and performance, including monitoring, scaling, and exploring emerging technologies. // Bio Misty Free Misty Free is a product manager at Jasper, where she focuses on supercharging marketers with speed and consistency in their marketing campaigns, with the power of AI. Misty has also collaborated with Stability and OpenAI to offer AI image generation within Jasper. She approaches product development with a "jobs-to-be-done" mindset, always starting with the "why" behind any need, ensuring that customer pain points are directly addressed with the features shipped at Jasper. In her free time, Misty enjoys crocheting amigurumi, practicing Spanish on Duolingo, and spending quality time with her family. Misty will be on a panel sharing her insights and experiences on the real-world use cases of LLMs. Davis Treybig Davis is a partner at Innovation Endeavors, an early-stage venture firm focused on teams solving hard technical & engineering problems. He personally focuses on computing infrastructure, AI/ML, and data. Dina Yerlan Head of Product, Generative AI Data at Adobe Firefly (family of foundation models for creatives). Artem Harutyunyan Artem is the Co-Founder & CTO at Bardeen AI. Prior to Bardeen, he was in engineering and product roles at Mesosphere and Qualys, and before that, he worked at CERN. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠Website: https://www.angellist.com/venture/relay Foundation by Isaac Asimov: https://www.amazon.com/Foundation-Isaac-Asimov/dp/0553293354 AngelList Relay blog: https://www.angellist.com/blog/introducing-angellist-relay --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Davis on LinkedIn: https://www.linkedin.com/in/davistreybig/ Connect with Misty on LinkedIn: https://www.linkedin.com/in/misty-miglorin/ Connect with Dina on LinkedIn: https://www.linkedin.com/in/dinayerlan/ Connect with Artem on LinkedIn: https://www.linkedin.com/in/artemharutyunyan/

MLOps Coffee Sessions #175 with Lamia Youseff, From Virtualization to AI Integration.

// Abstract

Lamia discusses how both Fortune 500 companies and SMBs lack the knowledge and capabilities to identify which use cases in their systems can benefit from AI integration. She emphasizes the importance of helping these companies integrate AI effectively and acquire the necessary capabilities to stay competitive in the market.

// Bio

By way of an introduction, Dr. Lamia Youseff has been working in AI / ML for ~25 years, first in academia (MIT, Stanford, UCSB), then large tech (Google, Microsoft, Apple, and Facebook), and most recently with startups in Generative AI. She is currently the executive director of JazzComputing, a Visiting Research Scientist at Stanford University in Computer Science and AI, and a research affiliate with MIT Computer Science and Artificial Intelligence Lab (CSAIL). Dr. Youseff earned her Ph.D. in computer science by studying computationally intensive workloads (such as AI / ML and HPC / Scientific Codes) and has built/led several AI teams as an executive and leader at large tech companies over the years (Google, Facebook, Microsoft, and Apple). She also earned her Master's in business management, strategy, and leadership from Stanford Graduate School of Business (GSB), where she is a guest lecturer today. Dr. Youseff regularly writes and speaks about AI and Machine Learning evolution at CIO/CTO/CEO summits.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Lamia on LinkedIn: https://www.linkedin.com/in/lyouseff/

Timestamps:

[00:00] Lamia's preferred coffee

[01:12] Takeaways

[03:00] Please like, share, and subscribe to our MLOps channels!

[03:20] Lamia's background

[09:52] Getting into Google Cloud

[13:10] The Google Cloud project

[16:38] The world before Kubernetes

[19:25] Evolution of virtualization

[23:20] Cloud evolution

[28:13] Kubernetes enables the ecosystem

[32:38] Multiple systems for machine learning

[34:40] Standardization for a greater good

[39:50] Complexity and pain points of ML in production

[46:26] JazzComputing

[50:33] Bridging gaps in AI implementation and investment

[51:19] Wrap up

MLOps Coffee Sessions #178 with LLMs in Production Conference part 2 LLM on K8s Panel, Manjot Pahwa, Rahul Parundekar, and Patrick Barker hosted by Outerbounds, Inc.'s Shrinand Javadekar. // Abstract Large Language Models require a new set of tools... or do they? K8s is a beast and we like it that way. How can we best leverage all the battle-hardened tech that K8s has to offer to make sure that our LLMs go brrrrrrr. Let's talk about it in this chat. // Bio Shrinand Javadekar Shri Javadekar is currently an engineer at Outerbounds, focussed on building a fully managed, large-scale platform for running data-intensive ML/AI workloads. Earlier, he spent time trying to start an MLOps company for which he was a co-founder and head of engineering. He led the design, development, and operations of Kubernetes-based infrastructure at Intuit, running thousands of applications, built by hundreds of teams and transacting billions of $$. He has been a founding engineer of the Argo open-source project and also spent precious time at multiple startups that were acquired by large organizations like EMC/Dell and VMWare. Manjot Pahwa Manjot is an investor at Lightspeed India and focuses on SaaS and enterprise tech. She has had an operating career of over a decade within the space of fintech, SaaS, and developer tools spanning various geos such as the US, Singapore, and India. Before joining Lightspeed, Manjot headed Stripe in India, successfully obtaining the payment aggregator license, growing the team from ~10 to 100+, and driving acquisitions in the region during that time. Rahul Parundekar Rahul has 13+ years of experience building AI solutions and leading teams. He is passionate about building Artificial Intelligence (A.I.) solutions for improving the Human Experience. He is currently the founder of A.I. Hero - a platform to help you fix and enrich your data with ML. At AI Hero, he has also been a big proponent of declarative MLOps - using Kubernetes to operationalize the training and serving lifecycle of ML models and has published several tutorials on his Medium blog. Before AI Hero, he was the Director of Data Science (ML Engineering) at Figure-Eight (acquired by Appen), a data annotation company, where he built out a data pipeline and ML model serving architecture serving 36 models (NLP, Computer Vision, Audio, etc.) and traffic of up to 1M predictions per day. Patrick Barker Patrick started his career in Big Data back when that was cool, then moved into Kubernetes near its inception. He has put major features into the Kubernetes API and built several platforms on top of it. In recent years he has moved into AI, with a focus on distributed machine learning. He is now working with a startup to reshape the world of AI agents. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠Website: https://www.angellist.com/venture/relay Foundation by Isaac Asimov: https://www.amazon.com/Foundation-Isaac-Asimov/dp/0553293354 AngelList Relay blog: https://www.angellist.com/blog/introducing-angellist-relay --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Shri on LinkedIn: https://www.linkedin.com/in/shrijavadekar/ Connect with Manjot on LinkedIn: https://www.linkedin.com/in/manjotpahwa/ Connect with Rahul on LinkedIn: https://www.linkedin.com/in/rparundekar/ Connect with Patrick on LinkedIn: https://www.linkedin.com/in/patrickbarkerco/

MLOps Coffee Sessions #174 with Michelle Marie Conway, Harnessing MLOps in Finance: Bringing Statistical Models to Life for Positive Impact, co-hosted by Stephen Batifol.

// Abstract

Michelle Marie Conway joins hosts Stephen Batifol and Demetrios to share their insights and experiences in the tech industry. Michelle emphasizes the importance of constant learning and adaptation in the rapidly changing tech industry. They discuss the need to stay up to date with the latest documentation, understand code logic, and be mindful when writing code. Michelle also reflects on their experiences as one of the few women in their university math class and often being the only woman on their team in the workplace. They discuss the need for more girls to pursue STEM subjects in schools and the importance of allies in the workplace. Additionally, Michelle explores the benefits and challenges of AI tools, sharing their experiences with tools like Gen AI and ChatGPT. While AI tools enhance productivity, Michelle also acknowledges the limitations of these tools in more technical tasks and the continued reliance on developer resources. This episode offers valuable insights into the importance of continuous learning, gender diversity in STEM, and the potential of AI tools in the field of MLOps.

// Bio

As an Irish woman who relocated to London after completing her university studies in Dublin, Michelle spent the past 12 years carving out a career in the data and tech industry. With a keen eye for detail and a passion for innovation, she has consistently leveraged my expertise to drive growth and deliver results for the companies she has worked for. As a dynamic and driven professional, Michelle is always looking for new challenges and opportunities to learn and grow, and she's excited to see what the future holds in this exciting and ever-evolving industry.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links⁠

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Stephen on LinkedIn: https://www.linkedin.com/in/stephen-batifol/

Connect with Michelle on LinkedIn: https://www.linkedin.com/in/michelle-conway-40337432

Timestamps:

[00:00] Michelle's preferred coffee

[02:04] Takeaways

[05:18] Please like, share, and subscribe to our MLOps channels!

[06:18] Michelle's journey in tech

[07:49] Engineering best practices

[09:38] Getting comfortable with the hump

[11:22] Clean coding fundamentals

[13:29] Working with the people

[14:09] GCP migration

[18:00] GCP migration length of journey

[18:38] Moving data focus

[19:18] Effectiveness of running 2 systems

[21:00] Dealing with discrepancies

[22:15] Using Nexus

[24:04] Migrating data from Teradata to BigQuery, strict security

[28:48] Hiring new people

[30:56] Securely managing financial data with millions of customers

[32:30] When things go wrong

[35:08] Finding the root cause

[36:28] Dealing with the producers' problems

[40:46] Rapid tech evolution, constant learning

[44:44] Teaching Python, using Gen AI for tasks

[46:34] Dealing with LLMs use cases

[49:15] Dealing with stakeholders and MLOps teams

[51:17] Having a translator

[52:18] Being a woman in the tech industry

[55:11] Encourage more girls in STEM, support women

[56:36] Women in the conversation on tech and female representation

[1:03:49] Wrap up

MLOps Coffee Sessions #176 with MLOps vs. LLMOps Panel, Willem Pienaar, Chris Van Pelt, Aparna Dhinakaran, and Alex Ratner hosted by Richa Sachdev. // Abstract What do MLOps and LLMOps have in common? What has changed? Are these just new buzzwords or is there validity in calling this ops something new? // Bio Richa Sachdev A passionate and impact-driven leader whose expertise spans leading teams, architecting ML and data-intensive applications, and driving enterprise data strategy. Richa has worked for a Tier A Start-up developing feature platforms and in financial companies, leading ML Engineering teams to drive data-driven business decisions. Richa enjoys reading technical blogs focussed on system design and plays an active role in the MLOps Community. Willem Pienaar Willem is the creator of Feast, the open-source feature store and a builder in the generative AI space. Previously Willem was an engineering manager at Tecton where he led teams in both their open source and enterprise initiatives. Before that Willem built the core ML systems and created the ML platform team at Gojek, the Indonesian decacorn. Chris Van Pelt Chris Van Pelt is a co-founder of Weights & Biases, a developer MLOps platform. In 2009, Chris founded Figure Eight/CrowdFlower. Over the past 12 years, Chris has dedicated his career optimizing ML workflows and teaching ML practitioners, making machine learning more accessible to all. Chris has worked as a studio artist, computer scientist, and web engineer. He studied both art and computer science at Hope College. Aparna Dhinakaran Aparna Dhinakaran is the Co-Founder and Chief Product Officer at Arize AI, a pioneer and early leader in machine learning (ML) observability. A frequent speaker at top conferences and thought leader in the space, Dhinakaran was recently named to the Forbes 30 Under 30. Before Arize, Dhinakaran was an ML engineer and leader at Uber, Apple, and TubeMogul (acquired by Adobe). During her time at Uber, she built several core ML Infrastructure platforms, including Michelangelo. She has a bachelor’s from UC Berkeley's Electrical Engineering and Computer Science program, where she published research with Berkeley's AI Research group. She is on a leave of absence from the Computer Vision Ph.D. program at Cornell University. Alex Ratner Alex Ratner is the co-founder and CEO at Snorkel AI, and an Affiliate Assistant Professor of Computer Science at the University of Washington. Prior to Snorkel AI and UW, he completed his Ph.D. in CS advised by Christopher Ré at Stanford, where he started and led the Snorkel open source project, and where his research focused on defining and forwarding the concept of “data-centric AI”, the idea that labeling and developing data is the new center of the AI development workflow. His academic work focuses on data-centric AI and related topics in data management and statistical learning techniques, and applications to real-world problems in medicine, science, and more. Previously, he earned his A.B. in Physics from Harvard University. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Richa on LinkedIn: https://www.linkedin.com/in/richasachdev/ Connect with Willem on LinkedIn: https://www.linkedin.com/in/willempienaar/ Connect with Chris on LinkedIn: https://www.linkedin.com/in/chrisvanpelt/ Connect with Aparna on Twitter: https://www.linkedin.com/in/aparnadhinakaran/ Connect with Alex on Twitter: https://www.linkedin.com/in/alexander-ratner-038ba239/

MLOps Coffee Sessions #173 with Beyang Liu, Building Cody, an Open Source AI Coding Assistant.

We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O

// Abstract

Root about the development of Cody, an open-source AI coding assistant. Cody empowers developers to query and comprehend code within codebases through the integration of robust language model capabilities. Sourcegraph tackles the intricacies of understanding intricate codebases by creating comprehensive code maps and employing AI for advanced search functionalities. Cody harnesses the potential of AI to offer features such as code exploration, natural language queries, and AI-powered code generation, augmenting developer productivity and code comprehension.

// Bio

Beyang Liu is the CTO and Co-founder of Sourcegraph. Prior to Sourcegraph, Beyang was an engineer at Palantir Technologies, building large-scale data analysis tools for Fortune 500 companies with large, complex codebases. Beyang studied computer science at Stanford, where he discovered his love for compilers and published some machine learning research as a member of the Stanford AI Lab.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related

Links⁠Website: https://beyang.com

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Beyang on LinkedIn: https://www.linkedin.com/in/beyang-liu/

Timestamps:

[00:00] Beyang's preferred coffee

[00:19] Takeaways

[01:25] Please like, share, and subscribe to our MLOps channels!

[01:48] Beyang background before Sourcegraph

[03:10] War stories

[04:30] Technological tool solution

[06:41] Landscape change in the past 10 years

[09:32] Code search engine evolution

[16:28] Vector databases

[17:40] Actual tech breakdown

[19:52] Incorporating AI into products amid organizational challenges

[25:39] Breaking down Cody

[28:04] Context fetching

[30:44] AI replicating human code understanding?

[36:22] Key for software creation

[40:26] Speak the language

[42:20] Leveraging LLMs

[44:18] Low code, no code movement

[47:54] Reliability issues amongst agents

[53:12] LLMs used in code and chat generation

[56:12] Dealing with rate limits and followers or failovers

[57:33] Unnecessary comparison

[1:00:26] Wrap up

MLOps Coffee Sessions #174 with Evaluation Panel, Amrutha Gujjar, Josh Tobin, and Sohini Roy hosted by Abi Aryan. We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O // Abstract Language models are very complex thus introducing several challenges in interpretability. The large amounts of data required to train these black-box language models make it even harder to understand why a language model generates a particular output. In the past, transformer models were typically evaluated using perplexity, BLEU score, or human evaluation. However, LLMs amplify the problem even further due to their generative nature thus making them further susceptible to hallucinations and factual inaccuracies. Thus, evaluation becomes an important concern. // Bio Abi Aryan Machine Learning Engineer @ Independent Consultant Abi is a machine learning engineer and an independent consultant with over 7 years of experience in the industry using ML research and adapting it to solve real-world engineering challenges for businesses for a wide range of companies ranging from e-commerce, insurance, education and media & entertainment where she is responsible for machine learning infrastructure design and model development, integration and deployment at scale for data analysis, computer vision, audio-speech synthesis as well as natural language processing. She is also currently writing and working in autonomous agents and evaluation frameworks for large language models as a researcher at Bolkay. Amrutha Gujjar CEO & Co-Founder @ Structured Amrutha Gujjar is a senior software engineer and CEO and co-founder of Structured, based in New York. With a Bachelor of Science in Computer Science from the University of Washington's Allen School of CSE, she brings expertise in software development and leadership to my work.

Connect with Amrutha on LinkedIn to learn more about her experience and discuss exciting opportunities in software development and leadership.

Josh Tobin

Founder @ GantryJosh Tobin is the founder and CEO of Gantry. Previously, Josh worked as a deep learning & robotics researcher at OpenAI and as a management consultant at McKinsey. He is also the creator of Full Stack Deep Learning (fullstackdeeplearning.com), the first course focused on the emerging engineering discipline of production machine learning. Josh did his PhD in Computer Science at UC Berkeley advised by Pieter Abbeel.

Sohini Roy

Senior Developer Relations Manager @ NVIDIASohini Bianka Roy is a senior developer relations manager at NVIDIA, working within the Enterprise Product group. With a passion for the intersection of machine learning and operations, Sohini specializes in the domains of MLOps and LLMOps. With her extensive experience in the field, she plays a crucial role in bridging the gap between developers and enterprise customers, ensuring smooth integration and deployment of NVIDIA's cutting-edge technologies. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abi on LinkedIn: https://www.linkedin.com/in/abiaryan/ Connect with Amrutha on LinkedIn: https://www.linkedin.com/in/amruthagujjar/ Connect with Josh on LinkedIn: https://www.linkedin.com/in/josh-tobin-4b3b10a9/ Connect with Sohini on Twitter: https://twitter.com/biankaroy_

MLOps Coffee Sessions #172 with Lingjiao Chen, FrugalGPT: Better Quality and Lower Cost for LLM Applications.

This episode is sponsored by QuantumBlack.

We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O

// Abstract

There is a rapidly growing number of large language models (LLMs) that users can query for a fee. We review the cost associated with querying popular LLM APIs, e.g. GPT-4, ChatGPT, J1-Jumbo, and find that these models have heterogeneous pricing structures, with fees that can differ by two orders of magnitude. In particular, using LLMs on large collections of queries and text can be expensive. Motivated by this, we outline and discuss three types of strategies that users can exploit to reduce the inference cost associated with using LLMs: 1) prompt adaptation, 2) LLM approximation, and 3) LLM cascade. As an example, we propose FrugalGPT, a simple yet flexible instantiation of LLM cascade that learns which combinations of LLMs to use for different queries in order to reduce cost and improve accuracy. Our experiments show that FrugalGPT can match the performance of the best individual LLM (e.g., GPT-4) with up to 98% cost reduction or improve the accuracy over GPT-4 by 4% with the same cost. The ideas and findings presented here lay a foundation for using LLMs sustainably and efficiently.

// Bio

Lingjiao Chen is a Ph.D. candidate in the Computer Science department at Stanford University. He is broadly interested in machine learning, data management, and optimization. Working with Matei Zaharia and James Zou, he is currently exploring the fast-growing marketplaces of artificial intelligence and data. His work has been published at premier conferences and journals such as ICML, NeurIPS, SIGMOD, and PVLDB, and partially supported by a Google fellowship.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links⁠Website: https://lchen001.github.io/

FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance paper: https://arxiv.org/abs/2305.05176

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Lingjiao on LinkedIn:

Timestamps:

[00:00] Lingjiao's preferred coffee

[00:35] Takeaways

[02:41] Sponsor Ad: Nayur Khan of QuantumBlack

[05:27] Lingjiao's research at Stanford

[07:51] Day-to-day research overview

[10:11] Inventing data management inspired abstractions research

[13:58] Agnostic Approach to Data Management

[15:56] Frugal GPT

[18:59] Just another data provider

[19:51] Frugal GPT breakdown

[26:33] First step of optimizing the prompts

[28:04] Prompt overlap

[29:06] Query Concatenation

[32:30] Money saving

[35:04] Economizing the prompts

[38:52] Questions to accommodate

[41:33] LLM Cascade

[47:25] Frugal GPT saves cost and improves performance

[51:37] End-user implementation

[52:31] Completion Cache

[56:33] Using a vector store

[1:00:51] Wrap up

MLOps Coffee Sessions #172 with LLMs in Production Conference part 2 Building LLM Products Panel, George Mathew, Asmitha Rathis, Natalia Burina, and Sahar Mor Using hosted by TWIML's Sam Charrington. We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O // Abstract There are key areas we must be aware of when working with LLMs. High costs and low latency requirements are just the tip of the iceberg. In this panel, we hear about common pitfalls and challenges we must keep in mind when building on top of LLMs. // Bio Sam Charrington Sam is a noted ML/AI industry analyst, advisor and commentator, and host of the popular TWIML AI Podcast (formerly This Week in Machine Learning and AI). The show is one of the most popular Tech podcasts with nearly 15 million downloads. Sam has interviewed over 600 of the industry’s leading machine learning and AI experts and has conducted extensive research into enterprise AI adoption, MLOps, and other ML/AI-enabling technologies. George Mathew George is a Managing Director at Insight Partners focused on venture-stage investments in AI, ML, Analytics, and Data companies as they are establishing product/market Fit. Asmitha Rathis Asmitha is a Machine Learning Engineer with experience in developing and deploying ML models in production. She is currently working at an early-stage startup, PromptOps, where she is building conversational AI systems to assist developers. Prior to her current role, she was an ML engineer at VMware. Asmitha holds a Master’s degree in Computer Science from the University of California, San Diego, with a specialization in Machine Learning and Artificial Intelligence. Natalia Burina Natalia is an AI Product Leader who was most recently at Meta, leading Responsible AI. During her time at Meta, she led teams working on algorithmic transparency and AI Privacy. In 2017 Natalia was recognized by Business Insider as “The Most Powerful Female Engineer in 2017”. Natalia was also an Entrepreneur in Residence at Foundation Capital, advising portfolio companies and working with partners on deal flow. Prior to this, she was the Director of Product for Machine Learning at Salesforce, where she led teams building a set of AI capabilities and platform services. Prior to Facebook and Salesforce, Natalia led product development at Samsung, eBay, and Microsoft. She was also the Founder and CEO of Parable, a creative photo network bought by Samsung in 2015. Natalia started her career as a software engineer after pursuing Bachelor's degree in Applied and Computational Mathematics from the University of Washington. Sahar Mor Sahar is a Product Lead at Stripe with 15y of experience in product and engineering roles. At Stripe, he leads the adoption of LLMs and the Enhanced Issuer Network - a set of data partnerships with top banks to reduce payment fraud. Prior to Stripe he founded a document intelligence API company, was a founding PM in a couple of AI startups, including an accounting automation startup (Zeitgold, acq'd by Deel), and served in the elite intelligence unit 8200 in engineering roles. Sahar authors a weekly AI newsletter (AI Tidbits) and maintains a few open-source AI-related libraries (https://github.com/saharmor). // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/

MLOps Coffee Sessions #171 with Thibaut Labarre, Using Large Language Models at AngelList, co-hosted by Ryan Russon.

We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O

// Abstract

Thibaut innovatively addressed previous system constraints, achieving scalability and cost efficiency. Leveraging AngelList investing and natural language processing expertise, they refined news article classification for investor dashboards. Central is their groundbreaking platform, AngelList Relay, automating parsing and offering vital insights to investors. Amid challenges like Azure OpenAI collaboration and rate limit solutions, Thibaut reflects candidly. The narrative highlights prompt engineering's strategic importance and empowering domain experts for ongoing advancement.

// Bio

Thibaut LaBarre is an engineering lead with a background in Natural Language Processing (NLP). Currently, Thibaut focuses on unlocking the potential of Large Language Model (LLM) technology at AngelList, enabling everyone within the organization to become prompt engineers on a quest to streamline and automate the infrastructure for Venture Capital. Prior to that, Thibaut began his journey at Amazon as an intern, where he built Heartbeat, a state-of-the-art NLP tool that consolidates millions of data points from various feedback sources, such as product reviews, customer contacts, and social media, to provide valuable insights to global product teams. Over the span of seven years, he expanded his internship project into an organization of 20 engineers. He received a M.S. in Computational Linguistics from the University of Washington.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links⁠

Website: https://www.angellist.com/venture/relay

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Ryan on LinkedIn: https://www.linkedin.com/in/ryanrusson/

Connect with Thibaut on LinkedIn: https://www.linkedin.com/in/thibautlabarre/

Timestamps:

[00:00] Thibaut's preferred beverage

[00:50] Takeaways

[04:05] Please like, share, and subscribe to our MLOps channels!

[04:44] A huge fan of Isaac Asimov

[07:20] Thibaut Labarre background

[09:13] AngelList as an organization

[10:50] AI sense of building

[12:29] System trade-offs

[15:20] OpenAI's limitation

[16:31] Human in the loop

[17:22] Classifying relevance

[18:09] Fight for value

[19:37] Added value

[22:10] Exploring efficient ways to automate tasks.

[24:20] Investing in off-the-shelf models

[27:56] AngelList Relay

[30:49] News article and investment document classification technology

[32:39] Back-end tech

[34:09] Prompt layer

[35:28] Prompt layer as a living

[37:04] Foreseeing no human intervention

[39:00] Blocking hallucinations

[40:33] Challenges

[43:49] Investments in other models besides OpenAI

[45:20] Integration with other models

[46:28] Ethical concerns when

[48:37] OpenAI breaking Prompts

[50:46] Wrap up

Sean Morgan is an active open-source contributor and maintainer and is the special interest group lead for TensorFlow Addons. Learn more about the platform for end-to-end AI Security at https://protectai.com/.

MLSecOps is Fundamental to Robust AI Security Posture Management (AISPM) // MLOps Podcast #257 with Sean Morgan, Chief Architect at Protect AI.

// Abstract

MLSecOps, which is the practice of integrating security practices into the AIML lifecycle (think infusing MLOps with DevSecOps practices), is a critical part of any team’s AI Security Posture Management. In this talk, we’ll discuss how to threat model realistic AIML security risks, how you can measure your organization’s AI Security Posture, and most importantly, how you can improve that security posture through the use of MLSecOps.

// Bio

Sean Morgan is the Chief Architect at Protect AI. In prior roles, he's led production AIML deployments in the semiconductor industry, evaluated adversarial machine learning defenses for DARPA research programs, and most recently scaled customers on interactive machine learning solutions at AWS. In his free time, Sean is an active open-source contributor and maintainer and is the special interest group lead for TensorFlow Addons.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Sean's GitHub: https://github.com/seanpmorgan

MLSecOps Community: https://community.mlsecops.com/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Sean on LinkedIn: https://www.linkedin.com/in/seanmorgan/

Timestamps:

[00:00] Sean's preferred coffee

[00:10] Takeaways

[01:39] Register for the Data Engineering for AI/ML Conference now!

[02:21] KubeCon Paris: Emphasis on security and AI

[05:00] Concern about malicious data during the training process

[09:29] Model builders, security, pulling foundational models, nuances

[12:13] Hugging Face research on security issues

[15:00] Inference servers exposed; potential for attack

[19:45] Balancing ML and security processes for ease

[23:23] Model artifact security in enterprise machine learning

[25:04] Scanning models and datasets for vulnerabilities

[29:23] Ray's user interface vulnerabilities lead to attacks

[32:07] ML Flow vulnerabilities present significant server risks

[36:04] Data ops essential for machine learning security

[37:32] Prioritized security in model and data deployment

[40:46] Automated scanning tool for improved antivirus protection

[42:00] Wrap up

MLOps Coffee Sessions #170 with Phillip Carter, All the Hard Stuff with LLMs in Product Development.

We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O

// Abstract

Delve into challenges in implementing LLMs, such as security concerns and collaborative measures against attacks. Emphasize the role of ML engineers and product managers in successful implementation. Explore identifying leading indicators and measuring ROI for impactful AI initiatives.

// Bio

Phillip is on the product team at Honeycomb, where he works on a bunch of different developer tooling things. He's an OpenTelemetry maintainer -- chances are, if you've read the docs to learn how to use OTel, you've read his words. He's also Honeycomb's (accidental) prompt engineering expert by virtue of building and shipping products that use LLMs. In a past life, he worked on developer tools at Microsoft, helping bring the first cross-platform version of .NET into the world and grow to 5 million active developers. When not doing computer stuff, you'll find Phillip in the mountains riding a snowboard or backpacking in the Cascades.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links⁠

Website: https://phillipcarter.dev/

https://www.honeycomb.io/blog/improving-llms-production-observability

https://www.honeycomb.io/blog/hard-stuff-nobody-talks-about-llmhttps://phillipcarter.dev/posts/how-to-make-an-fsharp-code-fixer/

The "hard stuff" post: https://www.honeycomb.io/blog/hard-stuff-nobody-talks-about-llm

Our follow-up on iterating on LLMs in prod: https://www.honeycomb.io/blog/improving-llms-production-observability

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Phillip on LinkedIn: https://www.linkedin.com/in/phillip-carter-4714a135/

Timestamps:

[00:00] Phillip's preferred coffee

[00:33] Takeaways

[01:53] Please like, share, and subscribe to our MLOps channels!

[02:45] Phillip's background

[07:15] Querying Natural Language

[11:25] Function calls

[14:29] Pasting errors or traces

[16:30] Error patterns

[20:22] Honeycomb's Improvement cycle

[23:20] Prompt boxes rationale

[28:06] Prompt injection cycles

[32:11] Injection Attempt

[33:30] UI undervalued, charging the AI feature

[35:11] ROI cost

[44:26] Bridging ML and Product Perspective

[52:53] AI Model Trade-offs

[56:33] Query assistant

[59:07] Honeycomb is hiring!

[1:00:08] Wrap up

MLOps Coffee Sessions #169 with Barak Turovsky, MLOps at the Age of Generative AI.

Thanks to Weights & Biases for sponsoring this episode. Check out their new course on evaluating and fine-tuning LLMs at wandb.me/genai-mlops.course

// Abstract

The talk focuses on MLOps aspects of developing, training, and serving Generative AI/Large Language models

// Bio

Barak is an Executive in Residence at Scale Venture Partners, a leading Enterprise venture capital firm. Barak spent 10 years as Head of Product and User Experience for Languages AI and Google Translate teams within the Google AI org, focusing on applying cutting-edge Artificial Intelligence and Machine Learning technologies to deliver magical experiences across Google Search, Assistant, Cloud, Chrome, Ads, and other products. Previously, Barak spent 2 years as a product leader within the Google Commerce team. Most recently, Barak served as Chief Product Officer, responsible for product management and engineering at Trax, a leading provider of Computer Vision AI solutions for the Retail and Commerce industries. Prior to joining Google in 2011, Barak was Director of Products in Microsoft’s Mobile Advertising, Head of Mobile Commerce at PayPal, and Chief Technical Officer at an Israeli start-up. He lived more than 10 years in 3 different countries (Russia, Israel, and the US) and speaks three languages. Barak earned a Bachelor of Laws degree from Tel Aviv University, Israel, and a Master’s of Business Administration from the University of California, Berkeley.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Bio and links about Barak's work: https://docs.google.com/document/d/1E4Yrmt_Y57oTEYHQQDvt71XzSJ8Ew5WvscAQbHV4K3U/edit

Framework for evaluating Generative AI use cases: https://www.linkedin.com/pulse/framework-evaluating-generative-ai-use-cases-barak-turovsky/?trackingId=%2BMRxEZ9WTPCNH2JscILTeg%3D%3D

The Great A.I. Awakening: https://www.nytimes.com/2016/12/14/magazine/the-great-ai-awakening.html

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Barak on LinkedIn: https://www.linkedin.com/in/baraktur/

Timestamps:

[00:00] Barak's preferred coffee

[00:23] Barak Turovsky's background

[03:10] Please like, share, and subscribe to our MLOps channels!

[04:09] Getting into tech

[08:39] First wave of AI

[12:39] Building a product at scale and the challenges

[15:59] Framework for evaluating Generative AI use cases

[24:33] Machine trust adoption

[29:13] Wandb's new course

[31:10] Focus on achievable use cases for LLMs

[36:36] User feedback

[38:23] Disruption of entertainment and customer interactions

[46:14] Get new tools or work with your own distribution?

[47:57] Importance of data engineers

[53:28] ML Engineers Collaborate with Product[

56:13] Wrap up

MLOps Coffee Sessions #168 with Piotr Niedźwiedź, Experiment Tracking in the Age of LLMs, co-hosted by Vishnu Rachakonda.

// Abstract

Piotr shares his journey as an entrepreneur and the importance of focusing on core values to achieve success. He highlights the mission of Neptune to support ML teams by providing them with control and confidence in their models. The conversation delves into the role of experiment tracking in understanding and debugging models, comparing experiments, and versioning models. Piotr introduces the concept of prompt engineering as a different approach to building models, emphasizing the need for prompt validation and testing methods.

// Bio

Piotr is the CEO of neptune.ai. Day to day, apart from running the company, he focuses on the product side of things. Strategy, planning, ideation, getting deep into user needs and use cases. He really likes it.
Piotr's path to ML started with software engineering. Always liked math and started programming when he was 7. In high school, Piotr got into algorithmics and programming competitions and loved competing with the best. That got him into the best CS and Maths program in Poland which funny enough today specializes in machine learning.
Piotr did his internships at Facebook and Google and was offered to stay in the Valley. But something about being a FAANG engineer didn’t feel right. He had this spark to do more, build something himself. So with a few of his friends from the algo days, they started Codilime, a software consultancy, and later a sister company Deepsense.ai machine learning consultancy, where he was a CTO.
When he came to the ML space from software engineering, he was surprised by the messy experimentation practices, lack of control over model building, and a missing ecosystem of tools to help people deliver models confidently.
It was a stark contrast to the software development ecosystem, where you have mature tools for DevOps, observability, or orchestration to execute efficiently in production. And then, one day, some ML engineers from Deepsense.ai came to him and showed him this tool for tracking experiments they built during a Kaggle competition (which they won btw), and he knew this could be big.
He asked around, and everyone was struggling with managing experiments. He decided to spin it off as a VC-funded product company, and the rest is history.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://neptune.ai/blog/author/piotr-niedzwiedz

https://www.youtube.com/playlist?list=PLKePQLVx9tOfKFbg9GY2Anl41Be4T1-m5 
https://thesequence.substack.com/p/-piotr-niedzwiedz-neptunes-ceo-on

https://open.spotify.com/episode/2KEqTMAHODbPKdUEtlrhm7?si=ed862b2ac7534e39
https://www.linkedin.com/in/piotrniedzwiedz/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Piotr on LinkedIn: https://www.linkedin.com/in/piotrniedzwiedz/

Timestamps:

[00:00] Introduction to Piotr Niedźwiedź

[01:35] Please like, share, and subscribe to our MLOps channels!

[01:58] Wojciech Zaremba

[05:20] The Olympiad

[06:31] Building own company

[12:16] Talking outside Poland with the same passion

[13:45] Adapting with Neptune

[19:35] Core values focus

[22:02] Product Vision changes with advances

[29:36] Control and confidence

[30:05] Experiment tracking existing use cases

[37:25] Control pane

[38:59] Piotr's prediction

[43:20] WiFi issues around the world

[44:09] Wrap up

MLOps Coffee Sessions #167 with Maxime Beauchemin, Treating Prompt Engineering More Like Code.

// Abstract

Promptimize is an innovative tool designed to scientifically evaluate the effectiveness of prompts. Discover the advantages of open-sourcing the tool and its relevance, drawing parallels with test suites in software engineering. Uncover the increasing interest in this domain and the necessity for transparent interactions with language models. Delve into the world of prompt optimization, deterministic evaluation, and the unique challenges in AI prompt engineering.

// Bio

Maxime Beauchemin is the founder and CEO of Preset, a Series B startup supporting and commercializing the Apache Superset project. Max was the original creator of Apache Airflow and Apache Superset when he was at Airbnb. Max has over a decade of experience in data engineering at companies like Lyft, Airbnb, Facebook, and Ubisoft.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Max's first MLOps Podcast episode: https://go.mlops.community/KBnOgN

Test-Driven Prompt Engineering for LLMs with Promptimize blog: https://maximebeauchemin.medium.com/mastering-ai-powered-product-development-introducing-promptimize-for-test-driven-prompt-bffbbca91535

https://maximebeauchemin.medium.com/mastering-ai-powered-product-development-Test-Driven Prompt Engineering for LLMs with Promptimize podcast: https://talkpython.fm/episodes/show/417/test-driven-prompt-engineering-for-llms-with-promptimize

Taming AI Product Development Through Test-driven Prompt Engineering // Maxime Beauchemin // LLMs in Production Conference lightning talk: https://home.mlops.community/home/videos/taming-ai-product-development-through-test-driven-prompt-engineering

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Max on LinkedIn: https://www.linkedin.com/in/maximebeauchemin/

Timestamps:

[00:00] Max introduces the Apache Superset project at Preset

[01:04] Max's preferred coffee

[01:16] Airflow creator

[01:45] Takeaways

[03:53] Please like, share, and subscribe to our MLOps channels!

[04:31] Check Max's first MLOps Podcast episode

[05:20] Promptimize

[06:10] Interaction with API

[08:27] Deterministic evaluation of SQL queries and AI

[12:40] Figuring out the right edge cases

[14:17] Reaction with Vector Database

[15:55] Promptomize Test Suite

[18:48] Promptimize vision

[20:47] The open-source blood

[23:04] Impact of open source

[23:18] Dangers of open source

[25:25] AI-Language Models Revolution

[27:36] Test-driven design

[29:46] Prompt tracking

[33:41] Building Test Suites as Assets

[36:49] Adding new prompt cases to new capabilities

[39:32] Monitoring speed and cost

[44:07] Creating own benchmarks

[46:19] AI feature adding more value to the end users

[49:39] Perceived value of the feature

[50:53] LLMs costs

[52:15] Specialized model versus Generalized model

[56:58] Fine-tuning LLMs use cases

[1:02:30] Classic Engineer's Dilemma

[1:03:46] Build exciting tech that's available

[1:05:02] Catastrophic forgetting

[1:10:28] Prompt-driven development

[1:13:23] Wrap up

MLOps Coffee Sessions #166 with Roy Hasson & Santona Tuli, Eliminating Garbage In/Garbage Out for Analytics and ML.

// Abstract

Shift left data quality ownership and observability that makes it easy for users to catch bad data at the source and stop it from entering your analytics/ML stack.

// Bio

Santona Tuli

Santona Tuli, Ph.D., began her data journey through fundamental physics—searching through massive event data from particle collisions at CERN to detect rare particles. She’s since extended her machine learning engineering to natural language processing, before switching focus to product and data engineering for data workflow authoring frameworks. As a Python engineer, she started with the programmatic data orchestration tool, Airflow, helping improve its developer experience for data science and machine learning pipelines. Currently, at Upsolver, she leads data engineering and science, driving developer research and engagement for the declarative workflow authoring framework in SQL. Dr. Tuli is passionate about building, as well as empowering others to build, end-to-end data and ML pipelines, scalably.

Roy Hasson

Roy is the head of product at Upsolver, helping companies deliver high-quality data to their analytics and ML tools. Previously, Roy led product management for AWS Glue and AWS Lake Formation.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://royondata.substack.com/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community:

https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup:

https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more:

https://mlops.community/

Connect with Demetrios on LinkedIn:

https://www.linkedin.com/in/dpbrinkm/

Connect with Roy on LinkedIn:

https://www.linkedin.com/in/royhasson/

Connect with Santona on LinkedIn:

https://www.linkedin.com/in/santona-tuli/

Timestamps:

[00:00] Santona's and Roy's preferred coffee

[01:05] Santona's and Roy's background

[03:33] Takeaways

[05:49] Please like, share, and subscribe to our MLOps channels!

[06:42] Back story of having Santona and Roy on the podcast

[09:51] Santona's story

[11:37] Optimal tag teamwork

[16:53] Dealing with stakeholder needs

[26:25] Having mechanisms in place

[27:30] Building for data Engineers vs building for data scientists

[34:50] Creating solutions for users

[38:55] User experience holistic point of view

[41:11] Tooling sprawl is real

[42:00] LLMs' reliability

[45:00] Things I would have loved to learn five years ago

[49:46] Wrap up

MLOps Coffee Sessions #165 with Sammy Sidhu, Python Power: How Daft Embeds Models and Revolutionizes Data Processing.

// Abstract

Sammy shares his fascinating journey in the autonomous vehicle industry, highlighting his involvement in two successful startup acquisitions by Tesla and Toyota. He emphasizes his expertise in optimizing and distilling models for efficient machine learning, which he has incorporated into his new company, Eventual. The company's open-source offering, daf, focuses on tackling the challenges of unstructured and complex data. Sammy discusses the future of MLOps, machine learning, and data storage, particularly in relation to the retrieval and processing of unstructured data. The Eventual team is developing Daft, an open-source query engine that aims to provide efficient data storage solutions for unstructured data, offering features like governance, schema evolution, and time travel. The conversation sheds light on the innovative developments in the field and the potential impact on various industries.

// Bio

Sammy is a Deep Learning and systems veteran, holding over a dozen publications and patents in the space. Sammy graduated from the University of California, Berkeley, where he did research in Deep Learning and High Performance Computing. He then joined DeepScale as the Chief Architect and led the development of perception technologies for autonomous vehicles. During this time, DeepScale grew rapidly and was subsequently acquired by Tesla in 2019. Staying in Autonomous Vehicles, Sammy joined Lyft Level 5 as a Senior Staff Software Engineer, building out core perception algorithms as well as infrastructure for machine learning and embedded systems. Level 5 was then acquired by Toyota in 2021, adopting much of its work. Sammy is now CEO and Co-Founder at Eventual Building Daft, an open-source query engine that specializes in multimodal data.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://sammysidhu.com/

Check out Daft, our open-source query engine for multimodal data! https://www.getdaft.io/

Here are some talks/shows we have given about it:- PyData Global (Dec 2022): Large-scale image processing: https://www.youtube.com/watch?v=ol6IQUbyeDo&ab_channel=PyData

- Ray Meetup (March 2023): Distributed ML preprocessing + training on Ray https://www.youtube.com/watch?v=1MpEYlIlu7w&t=2972s&ab_channel=Anyscale

- The Data Stack Show (April 2023): Self-Driving Technology and Data Infrastructure with Sammy Sidhu https://datastackshow.com/podcast/the-prql-self-driving-technology-and-data-infrastructure-with-sammy-sidhu-co-founder-and-ceo-of-eventual/

Chain of Thought for LLMs: https://cobusgreyling.medium.com/chain-of-thought-prompting-in-llms-1077164edf97

Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes: https://arxiv.org/abs/2305.02301

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Sammy on LinkedIn: https://www.linkedin.com/in/sammy-sidhu/

MLOps Coffee Sessions #164 with Rob Hirschfeld, Open Source and Fast Decision Making. This episode is brought to you by.

// Abstract

Rob Hirschfeld, the CEO and co-founder of Rack N, discusses his extensive experience in the DevOps movement. He shares his notable achievement of coining the term "the cloud" and obtaining patents for infrastructure management and API provision. Rob highlights the stagnant progress in operations and the persistent challenges in security and access controls within the industry. The absence of standardization in areas such as Kubernetes and single sign-on complicates the development of robust solutions. To address these issues, Rob underscores the significance of open-source practices, automation, and version control in achieving operational independence and resilience in infrastructure management.

// Bio

Rob is the CEO and Co-founder of RackN, an Austin-based start-up that develops software to help automate data centers, which they call Digital Rebar. This platform helps connect all the different pieces and tools that people use to manage infrastructure into workflow pipelines through seamless multi-component automation across the different pieces and parts needed to bring up IT systems, platforms, and applications. Rob has a background in Scale Computing, Mechanical and Systems Engineering, and specializes in large-scale complex systems that are integrated with the physical environment. He has founded companies and been in the cloud and infrastructure space for nearly 25 years, and has done everything from building the first Clouds using ESXi betas to serving four terms on the OpenStack Foundation Board. Rob was trained as an Industrial Engineer and holds degrees from Duke University and Louisiana State University.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://rackn.com/

https://robhirschfeld.com/about/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Rob on LinkedIn: https://www.linkedin.com/in/rhirschfeld/

Timestamps:

[00:00] Rob's preferred coffee

[00:17] Rob Hirschfeld's background

[01:42] Takeaways

[02:36] Please like, share, and subscribe to this channel!

[03:09] Creation of Cloud

[08:38] Changes in Cloud after 25 Years

[10:54] Pros and cons of microservices

[13:06] Secure Access Provisioning

[15:46] Parallelism with ads

[18:08] Redfish protocol

[20:21] Impact of using open source vs using a SAS provider

[26:15] Automation

[32:39] Embrace Operational Flexibility

[35:08] Automating infrastructure inefficiently

[41:26] Legacy code and resiliency

[43:39] Collection of metadata

[45:50] RackN

[51:23] Granular Cloud Preferences

[54:35] Reframing of perceived complexity

[57:32] Generative DevOps

[58:50] Wrap up

MLOps Coffee Sessions #163 with Yujian Tang, Democratizing AI, co-hosted by Abi Aryan.

// Abstract

The popularity of ChatGPT has brought large language model (LLMs) apps and their supporting technologies to the forefront. One of the supporting technologies is vector databases. Yujian shares how vector databases like Milvus are used in production and how they solve one of the biggest problems in LLM app building - data issues. They also discuss how Zilliz is democratizing vector databases through education, expanding access to technologies, and technical evangelism.

// Bio

Yujian Tang is a Developer Advocate at Zilliz. He has a background as a software engineer working on AutoML at Amazon. Yujian studied Computer Science, Statistics, and Neuroscience with research papers published at conferences, including IEEE Big Data. He enjoys drinking bubble tea, spending time with family, and being near water.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Yujian on LinkedIn: https://www.linkedin.com/in/yujiantang

Timestamps:

[00:00] Yujian's preferred coffee

[02:40] Takeaways

[05:14] Please share this episode with your friends!

[06:39] Vector databases trajectory

[09:00] 2 start-up companies created by Yujian

[09:39] Uninitiated Vector Databases

[12:20] Vector Databases trade-off

[14:16] Difficulties in training LLMs

[23:30] Enterprise use cases

[27:38] Process/rules not to use LLMs unless necessary

[32:14] Setting up returns

[33:13] When not to use Vector Databases

[35:30] Elastic search

[36:07] Generative AI apps' common pitfalls

[39:35] Knowing your data

[41:50] Milvus

[48:28] Actual Enterprise use cases

[49:32] Horror stories

[50:31] Data mesh

[51:06] GPTCash

[52:10] Shout out to the Seattle Community!

[53:44] Wrap up

MLOps Coffee Sessions #162 with Soham Chatterjee, From LLMs to TinyML: The Dynamic Spectrum of MLOps, co-hosted by Abi Aryan.

// Abstract

Explore the spectrum of MLOps from large language models (LLMs) to TinyML. Soham highlights the difficulties of scaling machine learning models and cautions against relying exclusively on OpenAI's API due to its limitations. Soham is particularly interested in the effective deployment of models and the integration of IoT with deep learning. He offers insights into the challenges and strategies involved in deploying models in constrained environments, such as remote areas with limited power, and utilizing small devices like Arduino Nano.

// Bio

Soham leads the machine learning team at Sleek, where he builds tools for automated accounting and back-office management. As an electrical engineer, Soham has a passion for the intersection of machine learning and electronics, specifically TinyML/Edge Computing. He has several courses on MLOps and TinyMLOps available on Udacity and LinkedIn, with more courses in the works.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/

Connect with Soham on LinkedIn: https://www.linkedin.com/in/soham-chatterjee

Timestamps:

[00:00] Soham's preferred coffee

[01:49] Takeaways

[05:33] Please share this episode with

[07:02] Soham's background

[09:00] From electrical engineering to Machine Learning

[10:40] Deep learning, Edge Computing, and Quantum Computing

[11:34] Tiny ML

[13:29] Favorite area in Tiny ML chain

[14:03] Applications explored

[16:56] Operational challenges transformation

[18:49] Building with Large Language Models

[25:44] Most Optimal Model

[26:33] LLMs path

[29:19] Prompt engineering

[33:17] Migrating infrastructures to a new product

[37:20] Your success where others failed

[38:26] API Accessibility

[39:02] Reality about LLMs

[40:39] The Compression angle adds to the bias

[43:28] Wrap up

MLOps Coffee Sessions #161 with Tuhin Srivastava, The Long Tail of ML Deployment, co-hosted by Abi Aryan. This episode is brought to you by QuantumBlack.

// Abstract

Baseten is an engineer-first platform designed to alleviate the engineering burden for machine learning and data engineers. Tuhin's perspective, based on research with Stanford students, emphasizes the importance of engineers embracing the engineering aspects and considering them from a reproductive perspective.

// Bio

Tuhin Srivastava is the co-founder and CEO of Baseten. Tuhin has spent the better part of the last decade building machine learning-powered products and is currently working on empowering engineers to build production-grade services with machine learning.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

QuantumBlack: https://www.mckinsey.com/capabilities/quantumblack/contact-us

Baseten: https://www.baseten.co/

Baseten Careers: https://www.baseten.co/careers

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/

Connect with Tuhin on LinkedIn: https://www.linkedin.com/in/tuhin-srivastava-60601114/

Timestamps:

[00:00] Partnership with QuantumBlack

[00:16] Nayur Khan presenting QuantumBlack

[03:35] QuantumBlack is hiring!

[03:47] Tuhin's preferred coffee

[05:03] Takeaways

[07:00] Please share this episode with a friend!

[07:12] Comments/Reviews

[08:49] Tuhin's background

[10:13] Finance and Law common complaint culture

[11:40] Doing Machine Learning in 2010 - 2011

[14:31] Gum broad or the next company shape?

[16:33] Engineers need to learn machine learning

[20:18] Software engineers need to dig deeper

[24:49] Cambrian Explosion

[27:53] The Holy Trifecta

[28:54] Objective truth and prompting

[31:23] Limitations of LLMs

[35:37] Documentation challenges

[38:25] Baseten creating valuable models

[40:37] Advocate for Microservices or API-based solution

[42:54] Learning Git pains

[44:16] Baseten back ups

[48:00] Baseten is hiring!

[49:32] Wrap up

Harcharan Kabbay is a Data Scientist & AI/ML Engineer with Expertise in MLOps, Kubernetes, and DevOps, Driving End-to-End Automation and Transforming Data into Actionable Insights.

MLOps for GenAI Applications // MLOps Podcast #256 with Harcharan Kabbay, Lead Machine Learning Engineer at World Wide Technology.

// Abstract

The discussion begins with a brief overview of the Retrieval-Augmented Generation (RAG) framework, highlighting its significance in enhancing AI capabilities by combining retrieval mechanisms with generative models. The podcast further explores the integration of MLOps, focusing on best practices for embedding the RAG framework into a CI/CD pipeline. This includes ensuring robust monitoring, effective version control, and automated deployment processes that maintain the agility and efficiency of AI applications. A significant portion of the conversation is dedicated to the importance of automation in platform provisioning, emphasizing tools like Terraform. The discussion extends to application design, covering essential elements such as key vaults, configurations, and strategies for seamless promotion across different environments (development, testing, and production). We'll also address how to enhance the security posture of applications through network firewalls, key rotation, and other measures. Let's talk about the power of Kubernetes and related tools to aid a good application design. The podcast highlights the principles of good application design, including proper observability and eliminating single points of failure. I would share strategies to reduce development time by creating templates for GitHub repositories by application types to be reused, also templates for pull requests, thereby minimizing human errors and streamlining the development process.

// Bio

Harcharan is an AI and machine learning expert with a robust background in Kubernetes, DevOps, and automation. He specializes in MLOps, facilitating the adoption of industry best practices and platform provisioning automation. With extensive experience in developing and optimizing ML and data engineering pipelines, Harcharan excels at integrating RAG-based applications into production environments. His expertise in building scalable, automated AI systems has empowered the organization to enhance decision-making and problem-solving capabilities through advanced machine-learning techniques.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Harcharan's Medium - https://medium.com/@harcharan-kabbay

Data Engineering for AI/ML Conference: https://home.mlops.community/home/events/dataengforai

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Harcharan on LinkedIn: https://www.linkedin.com/in/harcharankabbay/locale=en_US

Timestamps:

[00:00] Harcharan's preferred coffee

[00:21] Takeaways

[01:03] Against local LLMs

[02:11] Creating bad habits

[02:42] Operationalizing RAG from the CI/CD perspective

[09:39] Kubernetes vs LLM Deployment

[12:12] Tool preferences in ML

[14:39] DevOps perspective of deployment

[17:44] Terraform Licensing Controversy

[22:47] PR Review Template Guidance

[27:32] People process tech order

[29:22] Register for the Data Engineering for AI/ML Conference now!

[30:00] ML monitoring strategies explained

[39:39] Serverless vs Overprovisioning

[44:43] Model SLA's and Monitoring

[51:04] LLM to App transition

[52:42] Ensuring Robust Architecture

[58:53] Chaos engineering in ML

[1:04:43] Wrap up

MLOps Coffee Sessions #160 with Matt Sharp, Data Developer at Shopify, Clean Code for Data Scientists, co-hosted by Abi Aryan.

// Abstract

Let's delve into Shopify's real-time serving platform, Merlin, which enables features like recommender systems, inbox classification, and fraud detection. Matt shares his insights on clean coding and the new book he is writing about LLMs in production.

// Bio

Matt is a Chemical Engineer turned Data scientist turned Data Engineer. Self-described "Recovering Data Scientist", Matt got tired of all the inefficiencies he faced as a Data Scientist and made the switch to Data Engineering. At Matt's last job, he ended up building the entire MLOps platform from scratch for a fintech startup called MX. Matt gives tips to data scientists on LinkedIn on how to level up their careers and has started to be known for my clean code tips in particular. Matt recently started a new job at Shopify.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/

Connect with Matt on LinkedIn: https://www.linkedin.com/in/matthewsharp/

Timestamps:

[00:00] Matt's preferred drink

[00:54] Takeaways

[03:04] Watch out for Matt's LLMs in Production book coming up!

[03:29] Please like, share, subscribe, and join the upcoming LLMs in Production Conference Part 2!

[05:07] Raising awareness about the fundamental problems of writing clean code

[07:57] Definition of clean code[09:46] Communicable clean code

[13:52] Getting out of Jupyter notebooks at the end of their life

[17:21] Exploratory data analysis

[21:22] Most popular post on LinkedIn

[26:41] Zilliz Ad

[27:44] Best practices on production-level software engineering

[29:41] Merlin

[33:51] Upcoming Shopify projects

[39:10] Matt's upcoming LLMs in Production book

[45:06] LLMs in Production book Early Access

[46:00] Wrap up

MLOps Coffee Sessions #159 with Maria Vechtomova, Lead ML engineer, and Basak Eskili, Machine Learning Engineer, at Ahold Delhaize. Why is MLOps Hard in an Enterprise? co-hosted by Abi Aryan.

// Abstract

MLOps is particularly challenging to implement in enterprise organizations due to the complexity of the data ecosystem, the need for collaboration across multiple teams, and the lack of standardization in ML tooling and infrastructure. In addition to these challenges, at Ahold Delhaize, there is a requirement for the reusability of models as our brands seek to have similar data science products, such as personalized offers, demand forecasts, and cross-sell.

// Bio

Maria Vechtomova

Maria is a Machine Learning Engineer at Ahold Delhaize. Maria is bridging the gap between data scientists, infra, and IT teams at different brands and focuses on standardization of machine learning operations across all the brands within Ahold Delhaize. During nine years in Data&Analytics, Maria tried herself in different roles, from data scientist to machine learning engineer, was part of teams in various domains, and has built broad knowledge. Maria believes that a model only starts living when it is in production. For this reason, last six years, her focus has been on the automation and standardization of processes related to machine learning.

Basak Eskili

Basak Eskili is a Machine Learning Engineer at Ahold Delhaize. She is working on creating new tools and infrastructure that enable data scientists to quickly operationalize algorithms. She is bridging the space between data scientists and platform engineers while improving the way of working in accordance with MLOps principles. In her previous role, she was responsible for bringing models to production. She focused on NLP projects and building data processing pipelines. Basak also implemented new solutions by using cloud services for existing applications and databases to improve time and efficiency.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related LinksMLOps Maturity Assessment Blog: https://mlops.community/mlops-maturity-assessment/

The Minimum Set of Must-Haves for MLOps Blog: https://mlops.community/the-minimum-set-of-must-haves-for-mlops/

Traceability & Reproducibility Blog: https://mlops.community/traceability-reproducibility/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/

Connect with Maria on LinkedIn: https://www.linkedin.com/in/maria-vechtomova/

Connect with Basak on LinkedIn: https://www.linkedin.com/in/ba%C5%9Fak-tu%C4%9F%C3%A7e-eskili-61511b58/

Timestamps:

[00:00] Maria & Basak's preferred coffee

[00:59] LLMs in Production Conference Part 2 coming up on June 15-16!

[02:08] Maria & Basak's background

[02:47] Takeaways

[04:52] A colorful history

[06:59] 4 levels of evolution

[08:15] Standardization and Model Registry Evolution

[11:52] Ahold Delhaize Standard task

[15:05] Ahold Delhaize Workflow

[25:19] Avoiding tooling sprawl

[28:10] Guardrails

[29:50] Secret sharing and credential sharing sloppy processes

[32:23] Distrust between DevOps engineers and data scientists

[33:29] MLOps vs DevOps

[35:31] Monitoring pieces heroes

[38:32] Future accumulative cost issues

[40:09] Exploratory phase in notebooks

MLOps Coffee Sessions #158 with Nils Reimer, MLOps Build or Buy, Large Language Model at Scale, co-hosted by Abi Aryan.

// Abstract

Large Language Models with billions of parameters have the possibility to change how we work with textual data. However, running them on scale at potentially hundreds of millions of texts a day is a massive challenge. Nils talks about finding the right model size for respective tasks, model distillation, and promising new ways of transferring knowledge from large to smaller models.

// Bio

Nils Reimers is highly recognized throughout the AI community for creating and maintaining the now-famous Sentence Transformers library (www.SBERT.net) used to develop, train, and use state-of-the-art LLMs. The project has 900+ stars on GitHub and 30M+ installations. Nils is currently the Director of Machine Learning at Cohere, where he leads the team that develops and trains Large Language Models (LLMs) with billions of parameters. Prior to Cohere, Nils created and led the science team for Neural Search at HuggingFace. Nils holds a Ph.D. in Computer Science from UKP in Darmstadt.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

(www.SBERT.net)

https://www.nils-reimers.de/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/

Connect with Nils on LinkedIn: https://www.linkedin.com/in/reimersnils/

Timestamps:

[00:00] Nils' preferred coffee

[00:45] Nils' background

[01:30] Takeaways

[06:47] Subscribe to our Newsletters and IRL Meetups, and leave your reviews!

[07:32] Nils' history

[10:39] From IT Security to Machine Learning

[13:22] Tangibility of IT and Security

[14:46] NLP transition

[15:55] Bad augmentation to new capabilities of LLMs

[16:59] Nils' concern during his PH.D.

[19:55] Making Money from Machine Learning

[22:06] Catastrophic forgetting

[26:34] Updating solutions

[28:42] Neural search space and building adaptive models

[31:23] Filtering models

[32:32] Latency issues

[36:53] Models running in parallel

[37:54] Generative models problems

[38:43] Nils' role at Cohere

[41:41] To build or not to build API

[43:00] Search models

[45:38] Large use cases

[46:43] Open source discussion within Cohere

[50:48] Competitive Edge

[55:27] Future world of API

[58:14] LLMs in Production Conference Part 2 announcement!

[1:00:17] Hopeful direction of Cohere's future

[1:02:33] Data silos

[1:04:34] Where to update the database and code

[1:05:24] Nils' focus

[1:08:49] Make money or save money

[1:10:30] Cohere's upcoming project

[1:12:37] Time spent red teaming the models

[1:14:05] Wrap up

We are having another LLMs in-production Virtual Conference. 50+ speakers combined with in-person activities around the world on June 15 & 16. Sign up free here: https://home.mlops.community/home/events/llm-in-prod-part-ii-2023-06-20 // Abstract This panel discussion is centered around a crucial topic in the tech industry - data privacy and security in the context of large language models and AI systems. The discussion highlights several key themes, such as the significance of trust in AI systems, the potential risks of hallucinations, and the differences between low and high-affordability use cases. The discussion promises to be thought-provoking and informative, shedding light on the latest developments and concerns in the field. We can expect to gain valuable insights into an issue that is becoming increasingly relevant in our digital world. // Bio Diego Oppenheimer Diego Oppenheimer is an entrepreneur, product developer, and investor with an extensive background in all things data. Currently, he is a Partner at Factory a venture fund specializing in AI investments as well as interim head of product at two LLM startups. Previously he was an executive vice president at DataRobot, Founder, and CEO at Algorithmia (acquired by DataRobot), and shipped some of Microsoft’s most used data analysis products including Excel, PowerBI, and SQL Server. Diego is active in AI/ML communities as a founding member and strategic advisor for the AI Infrastructure Alliance and MLops.Community and works with leaders to define ML industry standards and best practices. Diego holds a Bachelor's degree in Information Systems and a Masters degree in Business Intelligence and Data Analytics from Carnegie Mellon University Gevorg Karapetyan Gevorg Karapetyan is the co-founder and CTO of ZERO Systems where he oversees the company's product and technology strategy. He holds a Ph.D. in Computer Science and is the author of multiple publications, including a US Patent. Vin Vashishta C-level Technical Strategy Advisor and Founder of V Squared, one of the first data science consulting firms. Our mission is to provide support and clarity for our clients’ complete data and AI monetization journeys. Over a decade in data science and a quarter century in technology building and leading teams and delivering products with $100M+ in ARR. Saahil Jain Saahil Jain is an engineering manager at You.com. At You.com, Saahil builds search, ranking, and conversational AI systems. Previously, Saahil was a graduate researcher in the Stanford Machine Learning Group under Professor Andrew Ng, where he researched topics related to deep learning and natural language processing (NLP) in resource-constrained domains like healthcare. Prior to Stanford, Saahil worked as a product manager at Microsoft on Office 365. He received his B.S. and M.S. in Computer Science at Columbia University and Stanford University respectively. Shreya Rajpal Shreya is the creator of Guardrails AI, an open-source solution designed to establish guardrails for large language models. As a founding engineer at Predibase, she helped build the Applied ML and ML infra teams. Previously, she worked at Apple's Special Projects Group on cross-functional ML, and at Drive.ai building computer vision models.

MLOps Coffee Sessions #157 with Katrina Ni & Aaron Maurer, MLOps Build or Buy, Startup vs. Enterprise? co-hosted by Jake Noble of tecton.ai.

This episode is sponsored by Tecton - Check out their feature store to get your real-time ML journey started.

// Abstract

There are a bunch of challenges with building useful machine learning at a B2B software company like Slack, but we've built some cool use cases over the years, particularly around recommendations. One of the key challenges is how to train powerful models while being prudent stewards of our clients' essential business data, and how to do so while respecting the increasingly complex landscape of international data regulation.

// Bio

Katrina Ni

Katrina is a Machine Learning Engineer in Slack ML Services Team, where they build ML platforms and integrate ML, e.g., Recommend API, Spam Detection, across product functionalities. Prior to Slack, she was a Software Engineer in Tableau's Explain Data Team, where they built tools that utilize statistical models and propose possible explanations to help users inspect, uncover, and dig deeper into the viz. Aaron MaurerAaron is a senior engineering manager in the infra organization at Slack, managing both the machine learning team and the real-time services team. In six years at Slack, most of which Aaron spent as an engineer, He worked on the search ranking, recommendation, spam detection, performance anomaly detection, and many other ML applications. Aaron is also an advisor to Eppo, an experimentation platform. Prior to Slack, Aaroon worked as a data scientist at Airbnb, earned a Master's in statistics at the University of Chicago, and helped develop econometric models projecting the Obamacare rollout at Acumen LLC.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Jake on LinkedIn: https://www.linkedin.com/in/jakednoble/

Connect with Katrina on LinkedIn: https://www.linkedin.com/in/katrina-ni-660b2590/

Connect with Aaron on LinkedIn: https://www.linkedin.com/in/aaron-maurer-4003b638/

Timestamps:

[00:00] Aaron and Katrina's preferred coffee

[00:41] Recommender and System and Jake

[02:06] Takeaways

[05:38] Introduction to Aaron Maurer & Katrina Ni

[06:53] Aaron Maurer & Katrina Ni's Recommend API blog post

[08:36] 10-pole machine learning use case and Rex's use case

[10:14] Genesis of Slack's recommender system framework

[11:47] The Special Sauce

[12:58] Speaking the same language

[15:23] Use case sources

[17:08] Slack's feature engineering

[17:52] Main CTR models

[18:40] Data privacy

[21:33] Slack's recommendations problem

[22:09] Fine-tuning the generative models

[23:30] Cold start problem

[26:02] Underrated

[28:24] Baseline

[28:55] Cold sore space

[30:15] LLMs in Production Conference Part 2 announcement!

[32:32] Data scientists transition to ML

[33:35] Unicorns do exist!

[34:43] Diversity of skill set

[36:02] The future of ML

[38:34] Model Serving

[40:11] MLOps Maturity level

[43:06] AWS Analogy

[45:05] Primary difficulty

[48:07] Wrap up

Sign up for the next LLM in production conference here: https://go.mlops.community/LLMinprod

Watch all the talks from the first conference: https://go.mlops.community/llmconfpart1

// Abstract In this panel discussion, the topic of the cost of running large language models (LLMs) is explored, along with potential solutions. The benefits of bringing LLMs in-house, such as latency optimization and greater control, are also discussed. The panelists explore methods such as structured pruning and knowledge distillation for optimizing LLMs. OctoML's platform is mentioned as a tool for the automatic deployment of custom models and for selecting the most appropriate hardware for them. Overall, the discussion provides insights into the challenges of managing LLMs and potential strategies for overcoming them. // Bio Lina Weichbrodt Lina is a pragmatic freelancer and machine learning consultant that likes to solve business problems end-to-end and make machine learning or a simple, fast heuristic work in the real world. In her spare time, Lina likes to exchange with other people on how they can implement best practices in machine learning, talk to her at the Machine Learning Ops Slack: shorturl.at/swxIN. Luis Ceze Luis Ceze is Co-Founder and CEO of OctoML, which enables businesses to seamlessly deploy ML models to production making the most out of the hardware. OctoML is backed by Tiger Global, Addition, Amplify Partners, and Madrona Venture Group. Ceze is the Lazowska Professor in the Paul G. Allen School of Computer Science and Engineering at the University of Washington, where he has taught for 15 years. Luis co-directs the Systems and Architectures for Machine Learning lab (sampl.ai), which co-authored Apache TVM, a leading open-source ML stack for performance and portability that is used in widely deployed AI applications. Luis is also co-director of the Molecular Information Systems Lab (misl.bio), which led pioneering research in the intersection of computing and biology for IT applications such as DNA data storage. His research has been featured prominently in the media including New York Times, Popular Science, MIT Technology Review, and the Wall Street Journal. Ceze is a Venture Partner at Madrona Venture Group and leads their technical advisory board. Jared Zoneraich Co-Founder of PromptLayer, enabling data-driven prompt engineering. Compulsive builder. Jersey native, with a brief stint in California (UC Berkeley '20) and now residing in NYC. Daniel Campos Hailing from Mexico Daniel started his NLP journey with his BS in CS from RPI. He then worked at Microsoft on Ranking at Bing with LLM(back when they had 2 commas) and helped build out popular datasets like MSMARCO and TREC Deep Learning. While at Microsoft he got his MS in Computational Linguistics from the University of Washington with a focus on Curriculum Learning for Language Models. Most recently, he has been pursuing his Ph.D. at the University of Illinois Urbana Champaign focusing on efficient inference for LLMs and robust dense retrieval. During his Ph.D., he worked for companies like Neural Magic, Walmart, Qualtrics, and Mendel.AI and now works on bringing LLMs to search at Neeva. Mario Kostelac Currently building AI-powered products in Intercom in a small, highly effective team. I roam between practical research and engineering but lean more towards engineering and challenges around running reliable, safe, and predictable ML systems. You can imagine how fun it is in LLM era :). Generally interested in the intersection of product and tech, and building a differentiation by solving hard challenges (technical or non-technical). Software engineer turned into Machine Learning engineer 5 years ago.

MLOps Coffee Sessions #156 with Melissa Barr & Michael Mui, Machine Learning Education at Uber, co-hosted by Lina Weichbrodt.

// Abstract

Melissa and Michael discuss the education program they developed for Uber's machine learning platform service, Michelangelo, during a guest appearance on a podcast. The program teaches employees how to use machine learning both in general and specifically for Uber. The platform team can obtain valuable feedback from users and use it to enhance the platform. The course was designed using engineering principles, making it applicable to other products as well.

// Bio

Melissa Barr

Melissa is a Technical Program Manager for ML & AI at Uber. She is based in New York City. She drives projects across Uber’s ML platform, delivery, and personalization teams. She also built out the first version of the ML Education Program in 2021.

Michael Mui

Michael is a Staff Technical Lead Manager on Uber AI's Machine Learning Platform team. He leads the Distributed ML Training team, which focuses on building elastic, scalable, and fault-tolerant distributed machine learning libraries and systems used to power machine learning development productivity across Uber. He also co-leads Uber’s internal ML Education initiatives. Outside of Uber, Michael also teaches ML at the Parsons School of Design in NYC as an Adjunct Faculty (mostly for the museum passes!) and guest lectures at the University of California, Berkeley.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://www.uber.com/blog/ml-education-at-uber-program-design-and-outcomes

/https://www.uber.com/blog/ml-education-at-uber/

https://www.uber.com/en-PH/blog/ml-education-at-uber/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Melissa on LinkedIn: https://www.linkedin.com/in/melissabarr1/

Connect with Michael on LinkedIn: https://www.linkedin.com/in/michael-c-mui/

Connect with Lina on LinkedIn: https://www.linkedin.com/in/lina-weichbrodt-344a066a/

Timestamps:

[00:00] Melissa and Michael's preferred coffee

[01:51] Takeaways

[05:40] Please subscribe to our newsletters and leave reviews on our podcasts!

[06:18] Machine learning at Uber education program

[07:45] The Uber courses

[10:03] Tailoring the Uber education system

[12:27] Growing out of the ML-Ed platform efforts

[14:14] Expanding the ML Market Size

[15:23] Relationship evolution

[17:36] Reproducibility best practices

[21:46] Learning development timeline

[26:29] Courses' effectiveness evaluation

[29:57] Tracking Progress Challenge

[31:25] ML platforms for internal tools

[35:07] Impact of ML Education at Uber

[39:30] Recommendations to companies that want to start an ML-Ed platform

[41:12] Early ML Adoption Program

[42:11] Homegrown or home-built platform

[42:54] Feature creation for a course

[45:24] ML Education at Uber: Frameworks Inspired by Engineering Principles

[49:42] The Future of ML Education at Uber

[52:28] Unclear ways to spread ML knowledge

[54:20] Module for Generative AI and ChatGPT

[55:05] Measurement of success

[56:39] Wrap up

MLOps Coffee Sessions #155 with Matei Zaharia, The Birth and Growth of Spark: An Open Source Success Story, co-hosted by Vishnu Rachakonda.

// Abstract

We dive deep into the creation of Spark, with the creator himself, Matei Zaharia, Chief Technologist at Databricks. This episode also explores the development of Databricks' other open source home run, ML Flow, and the concept of "lake house ML". As a special treat, Matei talked to us about the details of the "DSP" (Demonstrate Search Predict) project, which aims to enable building applications by combining LLMs and other text-returning systems.

// About the guest:

Matei has the unique advantage of being able to see different perspectives, having worked in both academia and the industry. He listens carefully to people's challenges and excitement about ML and uses this to come up with new ideas. As a member of Databricks, Matei also has the advantage of applying ML to Databricks' own internal practices. He is constantly asking the question, "What's a better way to do this?"

// Bio

Matei Zaharia is an Associate Professor of Computer Science at Stanford and Chief Technologist at Databricks. He started the Apache Spark project during his Ph.D. at UC Berkeley, and co-developed other widely used open-source projects, including MLflow and Delta Lake, at Databricks. At Stanford, he works on distributed systems, NLP, and information retrieval, building programming models that can combine language models and external services to perform complex tasks. Matei’s research work was recognized through the 2014 ACM Doctoral Dissertation Award for the best Ph.D. dissertation in computer science, an NSF CAREER Award, and the US Presidential Early Career Award for Scientists and Engineers (PECASE).

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://cs.stanford.edu/~matei/https://spark.apache.org/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/

Connect with Matei on LinkedIn: https://www.linkedin.com/in/mateizaharia/

Timestamps:

[00:00] Matei's preferred coffee

[01:45] Takeaways

[05:50] Please subscribe to our newsletters, join our Slack, and subscribe to our podcast channels!

[06:52] Getting to know Matei as a person

[09:10] Spark

[14:18] Open and freewheeling cross-pollination

[16:35] Actual formation of Spark

[20:05] Spark and MLFlow Similarities and Differences

[24:24] Concepts in MLFlow

[27:34] DJ Khalid of the ML world

[30:58] Data Lakehouse

[33:35] Stanford's unique culture of the Computer Science Department

[36:06] Starting a company

[39:30] Unique advice to grad students

[41:51] Open source project

[44:35] LLMs in the New Revolution

[47:57] Type of company to start with

[49:56] Emergence of Corporate Research Labs

[53:50] LLMs size context

[54:44] Companies to respect

[57:28] Wrap up

MLOps Coffee Sessions #154 with Waleed Kadous, ML Scalability Challenges, co-hosted by Abi Aryan.

// Abstract

Dr. Waleed Kadous, Head of Engineering at Anyscale, discusses the challenges of scalability in machine learning and his company's efforts to solve them. The discussion covers the need for large-scale computing power, the importance of attention-based models, and the tension between big and small data.

// Bio

Dr. Waleed Kadous leads engineering at Anyscale, the company behind the open-source project Ray, the popular scalable AI platform. Prior to Anyscale, Waleed worked at Uber, where he led overall system architecture, evangelized machine learning, and led the Location and Maps teams. He previously worked at Google, where he founded the Android Location and Sensing team, responsible for the "blue dot" as well as ML algorithms underlying products like Google Fit.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: anyscale.comhttps://www.youtube.com/watch?v=hzW0AKKqew4

https://www.anyscale.com/blog/WaleedKadous-why-im-joining-anyscale

Ray Summit: https://raysummit.anyscale.com/

Anyscale careers: https://www.anyscale.com/careers

Learning Ray O'Reilly's book. It's free to anyone interested. https://www.anyscale.com/asset/book-learning-ray-oreilly

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/

Connect with Waleed on LinkedIn: https://www.linkedin.com/in/waleedkadous/

Timestamps:

[00:00] Waleed's preferred coffee

[00:38] Takeaways

[07:37] Waleed's background

[13:16] Nvidia investment with Rey

[14:00] Deep Learning use cases

[17:52] Infrastructure challenges

[22:01] MLOps level of maturity

[26:42] Scale overloading

[29:21] Large Language Models

[32:40] Balance between fine-tuning forces prompts engineering

[35:51] Deep Learning movement

[42:05] Open-source models have enough resources

[44:11] Ray[47:59] Value add for any scale from Ray

[48:55] "Big data is dead" reconciliation

[52:43] Causality in Deep Learning

[55:16] AI-assisted Apps

[57:59] Ray Summit is coming up in September!

[58:49] Anyscale is hiring!

[59:25] Wrap up

This exclusive podcast episode covers the key findings from the LLM in-production survey that we conducted over the past month.

For all the data to explore yourself use this link https://docs.google.com/spreadsheets/d/13wdBwkX8vZrYKuvF4h2egPh0LYSn2GQSwUaLV4GUNaU/edit?usp=sharing

Sign up for our LLM in-production conference happening on April 13th (TODAY) here:
https://home.mlops.community/home/events/llms-in-production-conference-2023-04-13

Nicolas Mauti is an MLOps Engineer from Lyon (France), Working at Malt.

BigQuery Feature Store // MLOps Podcast #255 with Nicolas Mauti, Lead MLOps at Malt.

// Abstract

Need a feature store for your AI/ML applications but overwhelmed by the multitude of options? Think again. In this talk, Nicolas shares how they solved this issue at Malt by leveraging the tools they already had in place. From ingestion to training, Nicolas provides insights on how to transform BigQuery into an effective feature management system.

We cover how Nicolas' team designed their feature tables and addressed challenges such as monitoring, alerting, data quality, point-in-time lookups, and backfilling. If you’re looking for a simpler way to manage your features without the overhead of additional software, this talk is for you. Discover how BigQuery can handle it all!

// Bio

Nicolas Mauti is the go-to guy for all things related to MLOps at Malt. With a knack for turning complex problems into streamlined solutions and over a decade of experience in code, data, and ops, he is a driving force in developing and deploying machine learning models that actually work in production. When he's not busy optimizing AI workflows, you can find him sharing his knowledge at the university. Whether it's cracking a tough data challenge or cracking a joke, Nicolas knows how to keep things interesting.

// MLOps Jobs board jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related LinksNicolas' Medium - https://medium.com/@nmauti

Data Engineering for AI/ML Conference: https://home.mlops.community/home/events/dataengforai

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Nicolas on LinkedIn: https://www.linkedin.com/in/nicolasmauti/?locale=en_US

Timestamps:

[00:00] Nicolas' preferred beverage

[00:35] Takeaways

[02:25] Please like, share, leave a review, and subscribe to our MLOps channels!

[02:57] BigQuery end goal

[05:00] BigQuery pain points

[10:14] BigQuery vs Feature Stores

[12:54] Freelancing Rate Matching issues

[16:43] Post-implementation pain points

[19:39] Feature Request Process

[20:45] Feature Naming Consistency

[23:42] Feature Usage Analysis

[26:59] Anomaly detection in data

[28:25] Continuous Model Retraining Process

[30:26] Model misbehavior detection

[33:01] Handling model latency issues

[36:28] Accuracy vs The Business

[38:59] BigQuery cist-benefit analysis

[42:06] Feature stores cost savings

[44:09] When not to use BigQuery

[46:20] Real-time vs Batch Processing

[49:11] Register for the Data Engineering for AI/ML Conference now!

[50:14] Wrap up

MLOps Coffee Sessions #153 with Rodolfo Núñez, Multilingual Programming and a Project Structure to Enable It, co-hosted by Abi Aryan.

// Abstract

It's really easy to mix different programming languages inside the same project and use a project template that enables easy collaboration. It's not about which language is better, but rather what language solves the given section of your problem better for you.

// Bio

Rodo has been working in the "Data Space" for almost 7 years. He was a Senior Data Scientist at Entel (a Chilean telecommunications company) and is now a Senior Machine Learning Engineer at the same company, where I also lead three mini teams dedicated to internal cybersecurity; design/promote continuous training for the entire Analytics team and also the whole company; and ensure the improvement of programming practices and code cleanliness standards.

Rodo is currently in charge of helping the team put models into production and define the tools that we will use for it. He specializes in R, but he's language/tool agnostic: you should use the tool that best solves your current problem.

Rodo studied Mathematical Engineering and MSc in Applied Mathematics at the University of Chile, in addition to General Engineering at the École Centrale Marseille.

Rodo really likes to share knowledge (bi-directionally) in whatever he thinks he can contribute. Some things that Rodo likes teaching are Data Science, Math, Latin Dances, and whatever he thinks he can give to people.

Rodo's other interests are computer games (especially Vermintide and Darktide), board games, and dancing to Latin rhythms. Also, he streams some games and Data science-related topics on Twitch.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://www.twitch.tv/en_coders

https://www.youtube.com/@en_coders

https://www.twitch.tv/rodonunez

https://github.com/rodo-nunez

https://github.com/en-coders-cl

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/

Connect with Rodo on LinkedIn: https://www.linkedin.com/in/rodonunez/

Timestamps:

[00:00] Rodo's preferred coffee

[00:16] Project structure

[00:34] Introduction to Rodolfo Núñez

[01:20] Takeaways

[04:34] Check out our Meetups, podcasts, newsletters, TikTok, and blog posts!

[05:50] Why data scientists should know how to code and code properly

[10:32] Becoming a team player

[14:02] Cookie-cutter project

[17:50] Markdown and Quarter over Jupyter notebooks

[23:18] Data scientists' templates

[30:06] Significance of scripts

[33:30] Monolith to Microservices

[34:33] Reproducibility

[36:37] Entire event processing scripts

[40:44] In-House cataloging solution

[42:08] Data flows

[46:00] Bonus topics!

[47:23] Elbow methodology

[50:17] Idea behind cross sampling

[50:51] Machine Learning and MLOps Security at Entel

[58:04] Wrap up

Worlds are colliding! This week, we join forces with the hosts of the Practical AI podcast to discuss all things machine learning operations. We talk about how the recent explosion of foundation models and generative models is influencing the world of MLOps, and we discuss related tooling, workflows, perceptions, etc.

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community:

https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup:

https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more:

https://mlops.community/

MLOps Coffee Sessions #152 with Keith Trnka, How A Manager Became a Believer in DevOps for Machine Learning.

// Abstract

Keith Trnka, a seasoned leader in the technology industry, set foot on the MLOps Podcast in a special episode where he shared insights into his experience leading data teams and machine learning teams, becoming a better software engineer, and overseeing a successful migration from a monolith to microservices in the healthcare sector without any downtime.

Keith's background includes directing data science at 98.6, improving language models at Swipe and Nuance, and completing a Ph.D. thesis in language modeling for assistive technology. His work in these areas has contributed to the development of technology applications for healthcare, including telemedicine visits using natural language processing and machine learning.

// Bio

Keith has been in the industry for about 11 years. Most recently, he was the Director of Data Science at 98point6, where we made telemedicine visits easier for doctors using natural language processing, machine learning, backend engineering, AWS, and frontend engineering. Prior to that, Keith improved the language models used in mobile phone keyboards at Swype and Nuance. And before that, He did his Ph.D. thesis in language modeling for assistive technology.

Currently, Keith is traveling, mentoring, and doing a side project on machine translation.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Keith on LinkedIn: https://www.linkedin.com/in/keith-trnka/

Timestamps:

[00:00] Keith's preferred coffee

[00:14] Introduction to Keith Trnka

[02:28] Getting into Machine Learning

[03:24] Learning applications

[07:26] Data scientists' pain working together

[12:32] Build: An Unorthodox Guide to Making Things Worth Making by Tony Fadell

[15:00] Context meaning

[17:08] Determining Healthcare workers' certain passions

[20:24] From Engineer to User Advocate

[23:13] Balancing the drive to explore and cutting losses

[27:12] Simplifying Machine Learning Process

[32:59] Hiring senior engineers

[38:30] High productivity team tips

[40:38] Inspire people not to dictate

[45:46] Code Review Aggressivity Check

[48:25] Zero Downtime Migration

[52:40] Choosing between Lambda and EKS

[55:23] Wrap up