Nullius in Verba – Details, episodes & analysis

Ioannidis, J. P. A. (2005). Why Most Published Research Findings Are False. PLoS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124

Neher, A. (1967). Probability Pyramiding, Research Error and the Need for Independent Replication. The Psychological Record, 17(2), 257–262. https://doi.org/10.1007/BF03393713

Moonesinghe, R., Khoury, M. J., & Janssens, A. C. J. W. (2007). Most Published Research Findings Are False—But a Little Replication Goes a Long Way. PLOS Medicine, 4(2), e28. https://doi.org/10.1371/journal.pmed.0040028

Stroebe, W. (2016). Are most published social psychological findings false? Journal of Experimental Social Psychology, 66, 134–144. https://doi.org/10.1016/j.jesp.2015.09.017

Diekmann, A. (2011). Are Most Published Research Findings False? Jahrbücher Für Nationalökonomie Und Statistik, 231(5–6), 628–635. https://doi.org/10.1515/jbnst-2011-5-606

Goodman, S., & Greenland, S. (2007). Why most published research findings are false: Problems in the analysis. PLoS Medicine, 4(4), e168.

Ioannidis, J. P. A. (2007). Why most published research findings are false: Author’s reply to Goodman and Greenland. PLoS Medicine, 4(6), e215.

In preparation for our discussion of "Why Most Published Research Findings Are False" by John Ioannidis from 2005, we read a very similar paper published 40 years earlier:

Neher, A. (1967). Probability Pyramiding, Research Error and the Need for Independent Replication. The Psychological Record, 17(2), 257–262. https://doi.org/10.1007/BF03393713

How I Fail. Blog by Veronika Cheplygina https://veronikach.com/category/how-i-fail/

Arkin, R. (2011). Most Underappreciated: 50 Prominent Social Psychologists Describe Their Most Unloved Work. Oxford University Press.

Kerr, N. L. (1998). HARKing: Hypothesizing After the Results are Known. Personality and Social Psychology Review, 2(3), 196–217. https://doi.org/10.1207/s15327957pspr0203_4

Sharpe, D. (2013). Why the resistance to statistical innovations? Bridging the communication gap. Psychological Methods, 18(4), 572–582. https://doi.org/10.1037/a0034177

Anti-Creativity Letters episode: https://nulliusinverba.podbean.com/e/prologus-23-the-anticreativity-letters-r-e-nisbett

Rouder, J. N., Haaf, J. M., & Snyder, H. K. (2019). Minimizing Mistakes in Psychological Science. Advances in Methods and Practices in Psychological Science, 2(1), 3–11. https://doi.org/10.1177/2515245918801915

Firestein, S. (2015). Failure: Why Science Is So Successful (First Edition). Oxford University Press.

In this episode, we delve into induction and deduction and talk further about issues related to generalizability. 

Shownotes

• Popper, K. The Logic of Scientific Discovery. (1953). Hutchinson &  Co. (Originally published in 1935)
• Yarkoni, T. (2022). The generalizability crisis. Behavioral and Brain Sciences, 45, e1.
• Mook, D. G. (1983). In defense of external invalidity. American psychologist, 38(4), 379-387.
• Salmon, W. C. (1981). Rational Prediction. The British Journal for the Philosophy of Science, 32(2), 115–125. https://doi.org/10.1093/bjps/32.2.115
• Reichenbach, H. (1938) [2006], Experience and Prediction: An Analysis of the Foundations and the Structure of Knowledge, Chicago: University of Chicago Press.
• Senn, S. (2007). Statistical issues in drug development (2nd ed). John Wiley & Sons.
• Ernst, M. D. (2004). Permutation Methods: A Basis for Exact Inference. Statistical Science, 19(4), 676–685. 
• Bacon, F. (1620). Instauratio magna [Novum organum]. London: John Bill.
• Urbach, P. (1982). Francis Bacon as a Precursor to Popper. The British Journal for the Philosophy of Science, 33(2), 113–132.

In this episode, we discuss the paper "In defense of external invalidity" by Douglas Mook. 

Shownotes

• Mook, D. G. (1983). In defense of external invalidity. American Psychologist, 38(4), 379–387.
• Mook, D. G. (1989). The myth of external validity. Everyday cognition in adulthood and late life, 25-43.
• The case of Phineas Gage was written up: Harlow, J. M. (1848). Passage of an iron rod through the head. The Boston Medical and Surgical Journal (1828-1851), 39(20)

A reading of the paper In Defense of External Invalidty by Douglas G. Mook, which will be discussed in the next episode. 

• Mook, D. G. (1983). In defense of external invalidity. American Psychologist, 38(4), 379–387. https://doi.org/10.1037/0003-066X.38.4.379

In this episode, we discuss the role of apprenticeship in training scientists and researchers. What’s the difference between traditional apprenticeship and cognitive apprenticeship? Does graduate training live up to its promise as an apprenticeship model? What can we do to improve the modeling of skills that are to be taught during graduate training? 

Shownotes

• Collins, A., Brown, J. S., & Holum, A. (1991). Cognitive apprenticeship: Making thinking visible. American educator, 15(3), 6-11.
• Gabrys, B. J., & Beltechi, A. (2012). Cognitive apprenticeship: The making of a scientist. In Reshaping doctoral education (pp. 144-155). Routledge.
• Casadevall, A., & Fang, F. C. (2016). Rigorous science: a how-to guide. MBio, 7(6), 10-1128.
• Alvesson, M., Gabriel, Y., & Paulsen, R. (2017). Return to meaning: A social science with something to say. Oxford University Press.
• Polanyi, M. (1958). Personal Knowledge: Towards a Post-Critical Philosophy (M. J. Nye, Ed.). University of Chicago Press.

This is a live episode, recorded in Växjö, Sweden (Linnaeus university) on September 24, 2024, at the 5th meeting of the Open Science Community Sweden and the Swedish Reproducibility Network. Thanks to André Kalmendal at Mono (https://monovaxjo.se) for recording the episode. 

In this episode, we discuss the paper "A case history in scientific method" by B. F. Skinner

Shownotes

• Skinner, B. F. (1956). A case history in scientific method. American psychologist, 11(5), 221.
• Richter, C. P. (1953). Free research versus design research. Science, 118(3056), 91–93.
• https://archive.org/details/WaldenTwoChapter01

In preparation for the next episode, in which we discuss this paper, here is a reading of: