Explore every episode of the podcast Total Survey Design
| Title | Pub. Date | Duration | |
|---|---|---|---|
| Designing Surveys for Multilingual Populations - Part 2. An Interview with Rubén Ángel Arias Rueda | 01 mai 2026 | 00:23:55 | |
In this episode of Total Survey Design, Dr. Azdren Coma sits down with Dr. Rubén Ángel Arias Rueda, Project Manager and Lead Researcher at Washington State University's Social and Economic Sciences Research Center, for the second part of a two-part series on multilingual surveys. Dr. Arias Rueda specializes in mixed-methods equity research, bilingual data collection, and community-engaged fieldwork, and has led statewide qualitative and survey studies for Washington State agencies and commissions—including research on opportunity gaps for Hispanic and Latino students and on agricultural labor conditions. Before joining the SESRC, he spent 15 years teaching Spanish and Spanish literature at all levels of the curriculum. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Designing Surveys for Multilingual Populations - Дизајнирање анкети за повеќејазични популации | 27 avr. 2026 | 00:10:33 | |
In this episode of Total Survey Design, Dr. Azdren Coma explores what it takes to design surveys for populations that speak two or more languages. The episode explains why measurement equivalence (ensuring responses mean the same thing across languages) matters more than literal translation, and why the goal should be conceptual equivalence rather than word-for-word matching. Drawing on a personal case study of the 2021 North Macedonia Census, Azdren walks through real-world translation pitfalls, including untranslated Cyrillic dropdown menus and grammatically awkward translations from Macedonian into Albanian, and closes with practical takeaways for reducing cognitive burden and bias across language groups. This is the first of a two-part series on surveying multilingual populations. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Personal Appeals in Contact Letters: How More Appeals Means More Responses | 05 avr. 2026 | 00:07:20 | |
In this episode, I explore research by Lena Le and Thom Allen at Washington State University on the use of personal appeals in survey contact letters — the persuasive statements that encourage people to participate, like explaining how results will be used or telling respondents their opinion matters. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| How the Dunning-Kruger Effect Skews Self-Reported Skill Ratings | 26 mars 2026 | 00:07:49 | |
In this episode, Dr. Azdren Coma explores how the Dunning-Kruger effect distorts self-reported skill ratings in hiring and surveys. A seemingly straightforward question like “On a scale of 0 to 10, how skilled are you in Microsoft Excel?” can produce misleading data because undefined scale points allow respondents to interpret them differently. Beginners often overestimate their abilities due to limited awareness of complexity, while more skilled individuals rate themselves more conservatively as they recognize what they don’t know—potentially reversing the ranking of candidates. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Writing Good Survey Questions - Part 3. Open-Ended Questions | 21 mars 2026 | 00:19:37 | |
In this episode, I will be talking about open-ended questions. Along with closed-ended questions, open-ended questions are the other major type of question used in surveys. This episode covers the purpose of open-ended questions, their pros and cons, the different types of open-ended questions, several examples, and finally some tips and best practices for implementing them effectively in surveys. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Question Stem Order Effects: Ice Cream, Ballots, and the Science of Being First | 11 mars 2026 | 00:13:16 | |
What happens when you are faced with a list of options but do not have a strong preference? In this episode, I explore how the simple order of a list can nudge a person's choice. I move beyond the idea of favorite flavors to look at the undecided voter and the hungry diner. I examine why being first on a ballot can sway an election by as much as 10 percent in some local races. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Intro to Survey Design for Marketing Research Students (Guest Lecture) | 04 mars 2026 | 00:39:39 | |
This episode is a live guest lecture recorded for a Marketing Research class at Washington State University. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Forced Choice vs Neutral Options: Why Forcing Choices May Damage Your Data | 20 févr. 2026 | 00:20:01 | |
In this episode of the Total Survey Design Podcast, I explore whether to include a neutral midpoint option in survey questions or force respondents to pick a side. I define forced-choice formats, from even-numbered scales and binary picks to pairwise comparisons. I discuss key evidence like the 2019 Pew Research experiment, which found forced yes/no questions increased reporting on sensitive topics compared to select-all-that-apply lists. I present strong arguments on both sides before explaining why I usually favor neutral options for attitude and opinion questions. I end by offering an elegant solution that combines the presence of a neutral option with the need for respondents to pick a side. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Forced Questions (Will Hurt Your Data) | 09 févr. 2026 | 00:06:32 | |
Should you ever force respondents to answer a survey question? In this episode of Total Survey Design, I examine forced questions. While these features are often used under the assumption that more complete data means better data, forcing responses frequently leads to worse outcomes by increasing survey breakoffs, encouraging satisficing, and undermining respondents’ trust. I outline more effective and respectful alternatives, encouraging responses without coercion. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Can Generative AI Help Design a Better Questionnaire? Feedback Help | 02 févr. 2026 | 00:14:26 | |
On this episode of the Total Survey Design podcast, I put generative AI to the test by asking it to critique and improve the Microsoft Word Feedback survey: a short, everyday questionnaire that many of us have seen after using the program. I compare the suggestions from Grok, ChatGPT, and Gemini, highlight where they agree, where they differ, and share my own thoughts on what they got right and what they missed. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Trump & Rogan vs. Survey Sampling: Addendum | 28 janv. 2026 | 00:05:04 | |
On this bonus episode of the Total Survey Design podcast, I follow up on Trump and Rogan’s polling critiques with a listener email highlighting nonresponse bias, the real issue behind low response rates, where responders often differ systematically from non-responders. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Trump & Rogan vs. Survey Sampling: What They Got Right, And Totally Wrong | 26 janv. 2026 | 00:16:35 | |
On this episode of the Total Survey Design podcast, I examine an episode of the Joe Rogan Experience podcast, #2219, which aired on October 25th, 2024, where he discusses survey sampling with President Donald Trump. They both express skepticism about the reliability of polls for making broad national claims. They also refute the ability to make predictions from small samples and criticize polling methodology. In this episode, I talk about what they got right and what they got wrong, diving into the science of sampling. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Designing a Feedback Survey From Scratch | 22 janv. 2026 | 00:15:55 | |
In this episode, Azdren creates a feedback survey from scratch, showing the process from idea formation, to messy first draft of the questionnaire, to final product. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Survey Weighting: When It Helps And When It Hurts | 08 déc. 2025 | 00:10:19 | |
This episode of the Total Survey Design podcast demystifies the statistical technique of survey weighting (or post-stratification), explaining how it corrects sample imbalances by giving "quiet groups" more influence. The hosts define weighting, detail when it's an effective correction tool for probability samples, and provide practical advice on avoiding common pitfalls like over-adjustment that can hurt data precision. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Nonresponse Error. Part 1 - Why Response Rate Does Not Necessarily Predict Nonresponse Error | 01 déc. 2025 | 00:05:26 | |
This episode explains the important distinction between response rate and nonresponse error in survey research. Drawing on principles from Dillman et al.(2014), it clarifies that a high response rate does not necessarily mean low nonresponse error, and vice versa. Through two practical examples, the episode illustrates how systematic differences between respondents and nonrespondents -- not the number of responses -- determine the presence of nonresponse error. The episode emphasizes that evaluating survey quality requires more than just looking at response rates; it requires understanding potential bias in who does and does not respond. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| A Conversation with Mark Miazga, Director of University Survey & Assessment Services at U of Minnesota | 23 nov. 2025 | 01:06:15 | |
In this episode, the hosts speak with Mark Miazga, Director of University Survey and Assessment Services at the University of Minnesota, about his journey into survey research. Mark shares how his upbringing in rural Wisconsin and a background in law and nonprofit work shaped his perspective on data and public service. The conversation covers his transition from corporate law to survey methodology, his work managing complex institutional surveys, and how empathy and practical experience influence his approach to data collection and analysis. The discussion also touches on bridging gaps between institutional research and survey methodology, as well as a discussion of an institution-wide survey calendar, survey timing, and sampling. The episode wraps up with four points of advice for future surveyors. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| A Conversation with Rob Santos, U.S. Census Bureau Director | 16 nov. 2025 | 00:46:12 | |
In this episode, we talk with Robert Santos, Director of the U.S. Census Bureau from 2022 to 2025, about his professional journey and philosophy toward data and public service. Rob shares how his upbringing in a Mexican American community in San Antonio and how his early exposure to music and math shaped his path into statistics. The conversation explores his commitment to equity in data, the importance of cultural understanding in survey design, and his reflections on his time at the Census. He also discusses challenges and innovations in modern data collection (including the use of AI), and the trade-offs between cost, quality, and timeliness in survey methodology. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| A Critique of Microsoft Word’s Feedback Survey | 03 août 2025 | 00:12:44 | |
In this episode, I talk about a survey I didn’t plan to analyze—one that popped up the moment I opened Microsoft Word. Microsoft asked for feedback, and well… I had some. I break down the flaws in their opening question, including directional bias, vague wording, and unlabeled scale points. I also reflect on how these design choices impact data quality and what could have been done better. It’s a reminder that even the most common surveys we see in the wild from even the biggest tech giants can teach us a lot about what not to do. If you just enjoy a good critique, you’ll want to stick around for this one. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| An Interview with Jon Krosnick on Survey Design, Public Opinion, and Scientific Rigor | 16 juin 2025 | 00:58:24 | |
In this episode of Total Survey Design, Dr. Azdren Coma and Dr. Seon Yup Lee interview Dr. Jon Krosnick, a leading expert on political behavior, survey methodology, and public opinion. Dr. Krosnick is a professor at Stanford University who has advised the U.S. Census Bureau, led national studies on voting behavior and climate change, and published extensively on how people form attitudes and make decisions. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| An Interview with Thom Allen | 24 mars 2025 | 01:00:51 | |
In this episode I sat down with Thom to talk about surveys. We talked about his time at the Social and Economic Sciences Research Center, working with Don Dillman on various projects including the 2000 U.S. Census, some of his favorite projects, my dissertation survey, the application of the Tailored Design Method on a wide variety of cultures, and much more. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Essential Considerations When Creating a Survey | 09 sept. 2024 | 00:12:14 | |
In this episode, we explore essential considerations for crafting effective surveys. We begin by emphasizing the importance of defining the purpose of your survey, which serves as a guide through each step of its creation. We discuss how understanding your main goal—whether it's measuring customer satisfaction or academic variables—shapes your questions and improves data quality. Additionally, we highlight the significance of knowing your target population, budget, and timeframe. By the episode's end, we offer practical advice on streamlining your survey to avoid unnecessary questions, reduce costs, and enhance response quality. Join us to learn how to design your survey with precision and purpose. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| The Target Population | 02 sept. 2024 | 00:07:53 | |
In this episode, we discuss the target population. We emphasize the importance of a well-defined target population, in context of the goal of your survey, and in relation to your sampling frame. We offer examples of target populations of various studies, and offer suggestions for the types of variables that you may want to consider when thinking about your target population. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| An Interview with James McCall, Survey Methodologist | 26 août 2024 | 00:28:05 | |
In this episode, we interview James McCall, a survey methodologist at Westat. James shares what first drew him to surveys, and what drove him into a career of surveys. We talk about his role as a survey methodologist in industry. We discuss cognitive testing, data scripts, and other elements of the art and science of survey design. We also discuss our pet-peeves, such as matrix questions, check-all-that-apply questions, pre-testing, having too-short timelines, and more. James also shares advice for social scientists who are interested in pursuing a career in survey methodology. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Design Elements - Progress Bars, Back Buttons, Forced Responses, and More! | 19 août 2024 | 00:27:09 | |
In this episode, we explore the key design elements of effective surveys, emphasizing the importance of user-friendly experiences and reliable data collection. But first, in our free talk segment we discuss the compensation structure of the YouGov platform. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| A Conversation About the Trust and Distrust of Surveys | 12 août 2024 | 00:21:59 | |
In this episode, we introduce a new free talk segment, where we discuss and celebrate Seon Yup's new job as chief survey methodologist at Kansas State University. We then explore why many people distrust surveys and some things we can do to regain their trust. We discuss key issues like sensitive topics, the impact of interviewer identity, and the prevalence of poorly designed surveys that fail to effect change. We also provide tips on recognizing legitimate surveys, emphasizing secure and ethically conducted inquiries. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| The Monetary Costs of Conducting Surveys - Part 1 | 04 août 2024 | 00:17:29 | |
In this episode, we explore the costs associated with conducting surveys. We start by debunking the common misconception that surveys are free, highlighting the significant expenses involved in various survey methods—from internet to mixed-mode surveys which are needed to collect useful data. We explore the astronomical costs of large-scale efforts like the U.S. Census, which can reach billions, and contrast these with more modest academic research surveys, which can still run into thousands of dollars. Special attention is given to the balance between cost and quality, underscoring that a higher investment often means better data. Through specific examples and a detailed breakdown of costs in both large and small-scale surveys, this episode provides a comprehensive overview of what it truly costs to gather reliable data. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Sample Design - Part 2. The Sampling Frame | 23 juil. 2024 | 00:19:53 | |
In our second episode focusing on sample design, we dive into the intricacies of creating a sampling frame. We explain what a sampling frame is and why it's crucial for ensuring your survey reaches the right target population. We discuss the four key elements of an ideal sampling frame: comprehensiveness, accuracy, specificity, and accessibility. Additionally, we highlight real-world examples and practical tips for researchers, from using address-based databases to leveraging online survey platforms. The episode concludes with brief phone interviews with listeners, emphasizing the potential biases in online panels and the importance of a well-constructed sampling frame. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Writing Good Survey Questions - Part 2. Response Options | 15 juil. 2024 | 00:09:39 | |
This episode is the second in a series of writing good survey questions, where we dive deep into the intricacies of response options for survey questions, a crucial component often overlooked in survey design. We explore the significance of well-constructed response options in enhancing data quality and the pitfalls of poorly designed ones that can lead to confusion. Our focus will primarily be on closed-ended questions, discussing the different types of scales like categorical and continuous, and how to balance the resolution of your options to capture necessary details without overwhelming respondents. Join us as we unpack strategies to make your survey options exhaustive, exclusive, and free from bias. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Writing Good Survey Questions - Part 1. Base Questions | 09 juil. 2024 | 00:11:54 | |
In this episode, we delve into the art of crafting effective survey questions. We discuss the importance of clear and precise wording, avoiding biases, and the impact of question formats on data quality. Learn practical tips for designing questions that yield accurate and meaningful data, whether for academic research, market analysis, or feedback collection. Join us as we explore examples and demonstrate how to refine a simple question into a well-crafted survey item. Perfect for anyone looking to improve their survey design skills! Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| The Total Survey Design Method - Reducing Costs, Increasing Benefits, and How to Get the Best Data! | 28 juin 2024 | 00:10:55 | |
In this episode, we explore the Total Survey Design Method to maximize data quality. Discover how reducing errors, rooted in social exchange theory, can enhance your survey's effectiveness. We'll delve into the three key factors—cost, benefit, and trust—discussing strategies to minimize respondent burden, highlight benefits, and establish trust. Learn practical tips on simplifying surveys, using incentives, and emphasizing legitimacy. Perfect for anyone aiming to improve survey response rates and data accuracy. Tune in for an in-depth look at creating high-quality surveys! Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Don Dillman – A Reflection in Memoriam | 19 juin 2024 | 00:43:10 | |
This episode is a tribute and commemoration of Don Dillman, who is known for his contributions to survey methodology, particularly the development of the Tailored Design Method (TDM) for conducting surveys. The episode reflects on his work as a professor and a mentor, his impact on our careers, and his legacy. Our unscripted conversation includes personal anecdotes, discussions on his methodologies, and a reflection on Don's influence on our work. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| The Four Sources of Survey Error | 07 juin 2024 | 00:10:35 | |
In this episode of the Total Survey Design Podcast, we explore the four main sources of survey error through the engaging story of Mario, a local pizzeria owner aiming to conduct an ethical survey. We delve into coverage error, sampling error, measurement error, and non-response error, explaining how each can distort survey results. By using Mario's quest to honestly determine if his pizzeria is the best in town, we illustrate practical strategies for reducing these errors, offering listeners valuable insights into improving survey accuracy and reliability. Join us as we unravel the complexities of total survey error and its impact on survey design. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Sample Design - Part 1. Sample Size - or why just over 1000 responses is enough to generalize findings to millions of people! | 03 juin 2024 | 00:13:18 | |
In this episode of the Total Survey Design Podcast, we explore the fascinating concept of sample size and its impact on survey accuracy. Discover why just about 1000 responses can represent the views of millions, debunking common misconceptions about survey sampling. We'll dive into the key parameters for calculating sample size, such as margin of error, confidence level, predicted variance, and target population size. Learn how a small, well-chosen sample can provide precise and reliable data, making large-scale surveys both practical and cost-effective. Tune in to understand the math behind the magic! Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| How to Lie with Surveys | 31 mai 2024 | 00:12:48 | |
In this episode, hosts Seon Yup Lee and Azdren Coma explore the ways surveys can mislead us, inspired by the book "How to Lie with Maps." They discuss common survey pitfalls, from bad question design to poor sampling, and provide real-world examples, such as misleading questions in political surveys and biased sampling in product ratings. This episode, the first in a series, aims to uncover the frequent errors in surveys that can skew data, whether by accident or intention. Tune in to learn how to spot and avoid these issues in your own survey designs. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| The Utility of Surveys | 19 mai 2024 | 00:08:02 | |
In this episode of the Total Survey Design Podcast, Seon Yup and Azdren dive into the utility of surveys and their vital role in understanding public opinion and decision-making. They share examples, like A&W's marketing mishap with their third-pound burger, and the importance of surveys in evaluating high-cost programs. From business insights to educational improvements, surveys offer invaluable data that can guide better choices and avoid costly mistakes. Tune in to learn why surveys matter and how they can transform various aspects of life and work. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Intro to the Podcast | 14 mai 2024 | 00:04:02 | |
In this debut episode of "Total Survey Design," hosts Azdren Coma and Seon Yup Lee introduce listeners to the intricacies and importance of surveys. Exploring the basics of survey creation and analysis, they discuss their professional backgrounds and the podcast's goals, including helping various audiences—from small business owners to academics—master survey design. They outline the upcoming season, highlighting topics such as survey utility, question design, and common survey issues, and tease special episodes on calculating sample sizes and interviewing expert guests. Join them to demystify surveys and enhance your understanding of this essential tool. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||
| Total Survey Design Trailer | 08 mai 2024 | 00:02:27 | |
Listen to this episode to get a preview of everything that the Total Survey Design podcast is all about. Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ | |||