AI Evals and Analytics Podcast – Details, episodes & analysis

Podcast details

Technical and general information from the podcast's RSS feed.

AI Evals and Analytics Podcast

Stella and Amy

News

Education

Business

Frequency: 1 episode/13d. Total Eps: 2

Build trustworthy AI products through evaluation-driven development.

Each episode covers practical evaluation strategies, industry trends, and best practices for building safe, reliable AI systems. From dataset generation and evals metrics design to cross-functional collaboration and post-launch analytics, we talk about how to build trustworthy and lasting AI products with a good AI evals and analytics framework.

Subscribe for practical techniques, industry insights, and guest interviews on AI evaluation and analytics.

More about AI Evals and Analytics -- https://ai-evals.org/

We (Stella & Amy) created the AI Evaluation & Analytics Playbook, a practical framework that helps teams ship production-ready, trustworthy AI systems.

Powered by Firstory Hosting

Recent rankings

Latest chart positions across Apple Podcasts and Spotify rankings.

Apple Podcasts

🇺🇸 USA - techNews
14/02/2026
#85
🇺🇸 USA - techNews
13/02/2026
#74
🇺🇸 USA - techNews
12/02/2026
#70
🇺🇸 USA - techNews
11/02/2026
#58
🇺🇸 USA - techNews
10/02/2026
#84
🇺🇸 USA - techNews
09/02/2026
#74
🇺🇸 USA - techNews
08/02/2026
#92
🇺🇸 USA - techNews
07/02/2026
#99
🇺🇸 USA - techNews
06/02/2026
#72
🇺🇸 USA - techNews
05/02/2026
#52

Spotify

No recent rankings available

Shared links between episodes and podcasts

Links found in episode descriptions and other podcasts that share them.

See all

https://www.linkedin.com/in/wenxingl/
4 shares
https://www.linkedin.com/in/amy17519/
4 shares

https://ai-evals.org/
5 shares
https://firstory.me/zh
3 shares

RSS feed quality and score

Technical evaluation of the podcast's RSS feed quality and structure.

See all

RSS feed quality

To improve

Score global : 38%

Publication history

Monthly episode publishing history over the past years.

Year

Latest published episodes

Recent episodes with titles, durations, and descriptions.

See all

Build AI Evals from Scratch: When and How?

Saturday, February 7, 2026 • Duration 17:48

What is Evaluation-driven development? When should you start building evals for your product? How to build it from scrach?

Using a real-world example of a customer chatbot for a medical insurance company, we walk through the process of setting up evals from scratch: translating product requirements into quantifiable metrics, curating quality test datasets (hint: you need fewer examples than you think), and making go/no-go decisions based on eval scores.

You'll learn why accuracy and safety require different approaches, how to avoid the trap of AI-generated test data, and why 94% vs 95% accuracy matters less than you'd expect—but safety guardrails are non-negotiable. This is the practical blueprint for anyone building AI products who wants to catch problems before users do.

00:00 – Introduction: Why We Need to Talk About Evals Now
00:39 – When to Start AI Evals?
03:20 – Example Setup: Medical Insurance Customer Chatbot
04:30 – Defining Evals in Product Requirements
07:19 – What Is Evaluation-Driven Development?
08:27 – Breaking Down "Accuracy": What Does It Really Mean?
09:42 – Dataset Curation: Quality Over Quantity
11:24 – How Big Should Your Test Set Be?
12:25 – Safety Guardrails: Knowledge Boundary and PII Leakage
15:29 – Making Release Decisions with Eval Metrics
17:33 – Start with What's Critical to Your Use Case

Stella Liu: https://www.linkedin.com/in/wenxingl/
Amy Chen: https://www.linkedin.com/in/amy17519/

More about AI Evals and Analytics -- https://ai-evals.org/

We (Stella & Amy) created the AI Evaluation & Analytics Playbook, a practical framework that helps teams ship production-ready, trustworthy AI systems.

Read moreCollapse

AI Evals Skills: Why Data Scientists Have a Natural Advantage

Monday, January 26, 2026 • Duration 22:10

What are the skills required for AI evals? Why data scientists have a natural advantage in AI evals?

Evaluating AI isn’t just about "vibe coding" with an AI assistant. It actually requires a solid foundation in statistics for picking sample sizes and coding to build your own testing frameworks. Data scientists have a huge head start here because they are already pros at designing metrics and communicating risks.

In the augural episode, we also explain why Evals (pre-launch testing) and Analytics (post-launch user feedback) are two sides of the same coin: one makes sure the AI works, and the other makes sure people actually love using it.

00:00 – Introduction to AI Evals & Analytics
01:31 – Why Data Scientists Have a Natural Advantage
01:59 – Technical Pillar: Statistics
02:48 – Technical Pillar: Coding & Prompt Engineering
05:03 – Technical Pillar: Dataset Generation
08:35 – Soft Skills & Stakeholder Collaboration
11:17 – Domain Expertise in Regulated Industries
15:50 – New Skills for the GenAI Era
19:25 – Why Evals and Analytics Must Come Together

Stella Liu: https://www.linkedin.com/in/wenxingl/
Amy Chen: https://www.linkedin.com/in/amy17519/

More about AI Evals and Analytics -- https://ai-evals.org/

We (Stella & Amy) created the AI Evaluation & Analytics Playbook, a practical framework that helps teams ship production-ready, trustworthy AI systems.

AI Evals and Analytics Podcast – Details, episodes & analysis

Podcast details

Recent rankings

Apple Podcasts

Spotify

Shared links between episodes and podcasts

RSS feed quality and score

Publication history

Latest published episodes

Related Shows Based on Content Similarities

Apple Podcasts

Spotify

Related Shows Based on Content Similarities

AI Evals and Analytics Podcast – Details, episodes & analysis

Podcast details

Recent rankings

Apple Podcasts

Spotify

Shared links between episodes and podcasts

LinkedIn2

Other2

RSS feed quality and score

Publication history

Latest published episodes

Similar podcasts and content

Related Shows Based on Content Similarities

Apple Podcasts

Spotify

Related Shows Based on Content Similarities

LinkedIn2

Other2