Deep Papers – Details, episodes & analysis

Podcast details

Technical and general information from the podcast's RSS feed.

Deep Papers

Arize AI

Science

Technology

Business

Frequency: 1 episode/18d. Total Eps: 60

Deep Papers is a podcast series featuring deep dives on today’s most important AI papers and research. Hosted by Arize AI founders and engineers, each episode profiles the people and techniques behind cutting-edge breakthroughs in machine learning.

Read moreCollapse

Site

RSS

Apple

Recent rankings

Latest chart positions across Apple Podcasts and Spotify rankings.

Apple Podcasts

🇨🇦 Canada - mathematics
22/08/2026
#40
🇬🇧 Great Britain - mathematics
22/08/2026
#11
🇩🇪 Germany - mathematics
22/08/2026
#64
🇺🇸 USA - mathematics
22/08/2026
#15
🇫🇷 France - mathematics
22/08/2026
#48
🇨🇦 Canada - mathematics
21/08/2026
#39
🇬🇧 Great Britain - mathematics
21/08/2026
#8
🇩🇪 Germany - mathematics
21/08/2026
#64
🇺🇸 USA - mathematics
21/08/2026
#73
🇫🇷 France - mathematics
21/08/2026
#48

Spotify

No recent rankings available

Shared links between episodes and podcasts

Links found in episode descriptions and other podcasts that share them.

See all

https://arize.com/community/
60 shares
https://arize.com/llm-evaluation/
57 shares
https://arize.com/ai-research-papers/
8 shares

https://twitter.com/arizeai
60 shares

RSS feed quality and score

Technical evaluation of the podcast's RSS feed quality and structure.

See all

RSS feed quality

To improve

Score global : 48%

Publication history

Monthly episode publishing history over the past years.

Year

Latest published episodes

Recent episodes with titles, durations, and descriptions.

See all

Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges

Friday, August 16, 2024 • Duration 39:05

This week’s paper presents a comprehensive study of the performance of various LLMs acting as judges. The researchers leverage TriviaQA as a benchmark for assessing objective knowledge reasoning of LLMs and evaluate them alongside human annotations which they find to have a high inter-annotator agreement. The study includes nine judge models and nine exam-taker models – both base and instruction-tuned. They assess the judge models’ alignment across different model sizes, families, and judge prompts to answer questions about the strengths and weaknesses of this paradigm, and what potential biases it may hold.

Read it on the blog: https://arize.com/blog/judging-the-judges-llm-as-a-judge/

Learn more about AI observability and evaluation, join the Arize AI Slack community or get the latest on LinkedIn and X.

Read moreCollapse

Breaking Down Meta's Llama 3 Herd of Models

Tuesday, August 6, 2024 • Duration 44:40

Meta just released Llama 3.1 405B–according to them, it’s “the first openly available model that rivals the top AI models when it comes to state-of-the-art capabilities in general knowledge, steerability, math, tool use, and multilingual translation.” Will the latest Llama herd ignite new applications and modeling paradigms like synthetic data generation? Will it enable the improvement and training of smaller models, as well as model distillation? Meta thinks so. We’ll take a look at what they did here, talk about open source, and decide if we want to believe the hype.

Read it on the blog: https://arize.com/blog/breaking-down-meta-llama-3/

Learn more about AI observability and evaluation, join the Arize AI Slack community or get the latest on LinkedIn and X.

Read moreCollapse

Reinforcement Learning in the Era of LLMs

Friday, March 15, 2024 • Duration 44:49

We’re exploring Reinforcement Learning in the Era of LLMs this week with Claire Longo, Arize’s Head of Customer Success. Recent advancements in Large Language Models (LLMs) have garnered wide attention and led to successful products such as ChatGPT and GPT-4. Their proficiency in adhering to instructions and delivering harmless, helpful, and honest (3H) responses can largely be attributed to the technique of Reinforcement Learning from Human Feedback (RLHF). This week’s paper, aims to link the research in conventional RL to RL techniques used in LLM research and demystify this technique by discussing why, when, and how RL excels.

Learn more about AI observability and evaluation, join the Arize AI Slack community or get the latest on LinkedIn and X.

Read moreCollapse

Sora: OpenAI’s Text-to-Video Generation Model

Friday, March 1, 2024 • Duration 45:08

This week, we discuss the implications of Text-to-Video Generation and speculate as to the possibilities (and limitations) of this incredible technology with some hot takes. Dat Ngo, ML Solutions Engineer at Arize, is joined by community member and AI Engineer Vibhu Sapra to review OpenAI’s technical report on their Text-To-Video Generation Model: Sora.

According to OpenAI, “Sora can generate videos up to a minute long while maintaining visual quality and adherence to the user’s prompt.” At the time of this recording, the model had not been widely released yet, but was becoming available to red teamers to assess risk, and also to artists to receive feedback on how Sora could be helpful for creatives.

At the end of our discussion, we also explore EvalCrafter: Benchmarking and Evaluating Large Video Generation Models. This recent paper proposed a new framework and pipeline to exhaustively evaluate the performance of the generated videos, which we look at in light of Sora.

Learn more about AI observability and evaluation, join the Arize AI Slack community or get the latest on LinkedIn and X.

Read moreCollapse

RAG vs Fine-Tuning

Thursday, February 8, 2024 • Duration 39:49

This week, we’re discussing "RAG vs Fine-Tuning: Pipelines, Tradeoff, and a Case Study on Agriculture." This paper explores a pipeline for fine-tuning and RAG, and presents the tradeoffs of both for multiple popular LLMs, including Llama2-13B, GPT-3.5, and GPT-4.

The authors propose a pipeline that consists of multiple stages, including extracting information from PDFs, generating questions and answers, using them for fine-tuning, and leveraging GPT-4 for evaluating the results. Overall, the results point to how systems built using LLMs can be adapted to respond and incorporate knowledge across a dimension that is critical for a specific industry, paving the way for further applications of LLMs in other industrial domains.

Learn more about AI observability and evaluation, join the Arize AI Slack community or get the latest on LinkedIn and X.

Read moreCollapse

Phi-2 Model

Friday, February 2, 2024 • Duration 44:29

We dive into Phi-2 and some of the major differences and use cases for a small language model (SLM) versus an LLM.

With only 2.7 billion parameters, Phi-2 surpasses the performance of Mistral and Llama-2 models at 7B and 13B parameters on various aggregated benchmarks. Notably, it achieves better performance compared to 25x larger Llama-2-70B model on multi-step reasoning tasks, i.e., coding and math. Furthermore, Phi-2 matches or outperforms the recently-announced Google Gemini Nano 2, despite being smaller in size.

Find the transcript and live recording: https://arize.com/blog/phi-2-model

Learn more about AI observability and evaluation, join the Arize AI Slack community or get the latest on LinkedIn and X.

Read moreCollapse

HyDE: Precise Zero-Shot Dense Retrieval without Relevance Labels

Friday, February 2, 2024 • Duration 36:22

We discuss HyDE: a thrilling zero-shot learning technique that combines GPT-3’s language understanding with contrastive text encoders.

HyDE revolutionizes information retrieval and grounding in real-world data by generating hypothetical documents from queries and retrieving similar real-world documents. It outperforms traditional unsupervised retrievers, rivaling fine-tuned retrievers across diverse tasks and languages. This leap in zero-shot learning efficiently retrieves relevant real-world information without task-specific fine-tuning, broadening AI model applicability and effectiveness.

Link to transcript and live recording: https://arize.com/blog/hyde-paper-reading-and-discussion/

Learn more about AI observability and evaluation, join the Arize AI Slack community or get the latest on LinkedIn and X.

Read moreCollapse

A Deep Dive Into Generative's Newest Models: Gemini vs Mistral (Mixtral-8x7B)–Part I

Wednesday, December 27, 2023 • Duration 47:50

For the last paper read of the year, Arize CPO & Co-Founder, Aparna Dhinakaran, is joined by a Dat Ngo (ML Solutions Architect) and Aman Khan (Product Manager) for an exploration of the new kids on the block: Gemini and Mixtral-8x7B.

There's a lot to cover, so this week's paper read is Part I in a series about Mixtral and Gemini. In Part I, we provide some background and context for Mixtral 8x7B from Mistral AI, a high-quality sparse mixture of experts model (SMoE) that outperforms Llama 2 70B on most benchmarks with 6x faster inference Mixtral also matches or outperforms GPT3.5 on most benchmarks. This open-source model was optimized through supervised fine-tuning and direct preference optimization.

Stay tuned for Part II in January, where we'll build on this conversation in and discuss Gemini-developed by teams at DeepMind and Google Research.

Link to transcript and live recording: https://arize.com/blog/a-deep-dive-into-generatives-newest-models-mistral-mixtral-8x7b/

Learn more about AI observability and evaluation, join the Arize AI Slack community or get the latest on LinkedIn and X.

Read moreCollapse

How to Prompt LLMs for Text-to-SQL: A Study in Zero-shot, Single-domain, and Cross-domain Settings

Monday, December 18, 2023 • Duration 44:59

We’re thrilled to be joined by Shuaichen Chang, LLM researcher and the author of this week’s paper to discuss his findings. Shuaichen’s research investigates the impact of prompt constructions on the performance of large language models (LLMs) in the text-to-SQL task, particularly focusing on zero-shot, single-domain, and cross-domain settings. Shuaichen and his team explore various strategies for prompt construction, evaluating the influence of database schema, content representation, and prompt length on LLMs’ effectiveness. The findings emphasize the importance of careful consideration in constructing prompts, highlighting the crucial role of table relationships and content, the effectiveness of in-domain demonstration examples, and the significance of prompt length in cross-domain scenarios.

Read the blog and watch the discussion: https://arize.com/blog/how-to-prompt-llms-for-text-to-sql-paper-reading/

Learn more about AI observability and evaluation, join the Arize AI Slack community or get the latest on LinkedIn and X.

Read moreCollapse

The Geometry of Truth: Emergent Linear Structure in LLM Representation of True/False Datasets

Thursday, November 30, 2023 • Duration 41:02

For this paper read, we’re joined by Samuel Marks, Postdoctoral Research Associate at Northeastern University, to discuss his paper, “The Geometry of Truth: Emergent Linear Structure in LLM Representation of True/False Datasets.” Samuel and his team curated high-quality datasets of true/false statements and used them to study in detail the structure of LLM representations of truth. Overall, they present evidence that language models linearly represent the truth or falsehood of factual statements and also introduce a novel technique, mass-mean probing, which generalizes better and is more causally implicated in model outputs than other probing techniques.

Find the transcript and read more here: https://arize.com/blog/the-geometry-of-truth-emergent-linear-structure-in-llm-representation-of-true-false-datasets-paper-reading/

Learn more about AI observability and evaluation, join the Arize AI Slack community or get the latest on LinkedIn and X.

Read moreCollapse

Deep Papers – Details, episodes & analysis

Podcast details

Recent rankings

Apple Podcasts

Spotify

Shared links between episodes and podcasts

RSS feed quality and score

Publication history

Latest published episodes

Related Shows Based on Content Similarities

Related Shows Based on Content Similarities

Apple Podcasts

Spotify

Deep Papers – Details, episodes & analysis

Podcast details

Recent rankings

Apple Podcasts

Spotify

Shared links between episodes and podcasts

Other59

Twitter3

RSS feed quality and score

Publication history

Latest published episodes

Similar podcasts and content

Related Shows Based on Content Similarities

Related Shows Based on Content Similarities

Other59

Twitter3

LinkedIn2

Apple Podcasts

Spotify