Everything Hertz – Details, episodes & analysis

Open access articles have democratized the availability of scientific research, but are author-paid publication fees undermining the quality of science? The preprint by Morgan and Smaldino - https://osf.io/preprints/osf/3ez9v Paul Smaldino's text book - Modeling social behavior (https://press.princeton.edu/books/paperback/9780691224145/modeling-social-behavior) Main edisode takeaways (AI-assisted summary) There is a wide variability in the quality of papers published in gold open access journals and a wide variate of open access journals, some of which prioritise quality research Diamond open access and green open access are alternative models to consider. The publishing industry needs more transparency and mandatory reporting of data. The pressure to publish more can lead to a crowding out problem and a focus on quantity over quality. Determining the quality of journals and papers is challenging, and there are varying levels of quality within different tiers of journals. Fraudulent publishing practices, such as paper mills and fake papers, can be facilitated by the market for publishing. The Publons service (R.I.P) and similar platforms can improve the transparency of peer review and provide a record of reviewers' contributions. Society journals may offer a better publishing model as they have a reputation to maintain and are less likely to prioritize quantity over quality. Other links Everything Hertz on social media - Dan on twitter (https://www.twitter.com/dsquintana) - James on twitter (https://www.twitter.com/jamesheathers) - Everything Hertz on twitter (https://www.twitter.com/hertzpodcast) - Everything Hertz on Facebook (https://www.facebook.com/everythinghertzpodcast/) Support us on Patreon (https://www.patreon.com/hertzpodcast) and get bonus stuff! $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month Citation Quintana, D. S., & Heathers, J. (2024, Sept 5). 184: A race to the bottom, Everything Hertz [Audio podcast], https://doi.org/10.17605/OSF.IO/3MUJV

Dan and James discuss a paper describing a journal editor's efforts to receive data from authors who submitted papers with results that seemed a little too beautiful to be true Main edisode takeaways (AI generated summary) * This editorial on the reproducibility crisis emphasizes the importance of providing raw data in scientific publications and highlights the need for transparency and accountability in the research process * The lack of oversight and the discrepancy between the amount of data required for scientific statements and what is often provided in academic publishing is a cause for concern. * Ensuring the integrity of scientific research requires the active involvement of editors, reviewers, and researchers in promoting transparency and upholding ethical standards. The scientific publishing process lacks oversight and accountability, leading to potential issues with the accuracy and trustworthiness of published papers. * Journals should prioritize maintaining high standards and ensuring that papers are thoroughly reviewed and validated before publication. * Changing behaviors within the scientific community, such as pledging to publish in open access journals, can promote positive change and improve research integrity. * There is a need for active maintenance and improvement of the systems and parameters of scientific research to prevent potential negative consequences. Links for papers we mentioned * The Molecular Brain editorial by Miyakawa: https://doi.org/10.1186/s13041-020-0552-2 * The STALT preprint: https://osf.io/6hste Other links Everything Hertz on social media - Dan on twitter (https://www.twitter.com/dsquintana) - James on twitter (https://www.twitter.com/jamesheathers) - Everything Hertz on twitter (https://www.twitter.com/hertzpodcast) - Everything Hertz on Facebook (https://www.facebook.com/everythinghertzpodcast/) Support us on Patreon (https://www.patreon.com/hertzpodcast) and get bonus stuff! $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month Citation Quintana, D. S., & Heathers, J. (2024, Aug 3). 183: Too beautiful to be true Everything Hertz [Audio podcast], https://doi.org/10.17605/OSF.IO/JF5MS

James proposes proposes a new type of consortium paper that could provide collaborative opportunities for researchers from countries that are underrepresented in published research papers. We also talk about computational reproducibility and paper publication bonuses. Links The paper from Steve Lindsay on computational reproducbility: A Plea to Psychology Professional Societies that Publish Journals: Assess Computational Reproducibility (https://doi.org/10.15626/MP.2023.4020) Other links Everything Hertz on social media - Dan on twitter (https://www.twitter.com/dsquintana) - James on twitter (https://www.twitter.com/jamesheathers) - Everything Hertz on twitter (https://www.twitter.com/hertzpodcast) - Everything Hertz on Facebook (https://www.facebook.com/everythinghertzpodcast/) Support us on Patreon (https://www.patreon.com/hertzpodcast) and get bonus stuff! $1 per month: A 20% discount on Everything Hertz merchandise, access to the occasional bonus episode, and the the warm feeling you're supporting the show $5 per month or more: All the stuff you get in the one dollar tier PLUS a bonus episode every month Citation Quintana, D.S., Heathers, J.A.J. (Hosts). (2023, October 31) "174: Smug missionaries with test tubes", Everything Hertz [Audio podcast], DOI: 10.17605/OSF.IO/FBHRZ

We chat with Amy Orben, who applies "multiverse" methodology to combat and expose analytical flexibility in her research area of the impact of digital technologies on psychological wellbeing. We also discuss ReproducibiliTea, an early career researcher-led journal club initiative she co-founded, which helps young researchers create local open science groups. Here are some more details and links: * The tweet (https://twitter.com/_vdeni_/status/1126485212337143808) pointing our Dan's gramatical error in his usual introduction. THANKS DENIS * Is Twitter melting our brains? * The history of "new technology" panic * What's the next panic? * Moral entrepreneurs: profiting from moral panic * Specification curve analysis (https://socialsciences.nature.com/users/200472-amy-orben/posts/42763-beyond-cherry-picking): a way to run all theoretically defensible analysis options on a given dataset * Amy's Nature Human Behavior paper (https://www.nature.com/articles/s41562-018-0506-1) * Amy's PNAS paper (https://www.pnas.org/content/early/2019/04/30/1902058116) * The longitudincal effect of social media use on life satisfaction * How should scientists speak out against dodgy science? * The story behind Reproducabilitea * The ReproducibiliTea podcast (https://soundcloud.com/reproducibilitea) * ReproducibiliTea stickers (https://twitter.com/OrbenAmy/status/1125712657334571008)! * The UK Reproducibility network (https://www.bristol.ac.uk/psychology/research/ukrn/about/) * Daniel Lakens' Coursera course (https://www.coursera.org/learn/statistical-inferences) * A multiverse of multiverses (https://osf.io/9qke2/) * Press releasing every paper might not be the best idea * Amy's book recommendation: The long way to a small angry planet (https://www.amazon.com/Long-Small-Angry-Planet-Wayfarers/dp/0062444131 https://www.amazon.com/Long-Small-Angry-Planet-Wayfarers/dp/0062444131) Other links - Amy on Twitter (https://twitter.com/OrbenAmy) - Dan on twitter (www.twitter.com/dsquintana) - James on twitter (www.twitter.com/jamesheathers) - Everything Hertz on twitter (www.twitter.com/hertzpodcast) - Everything Hertz on Facebook (www.facebook.com/everythinghertzpodcast/) Music credits: Lee Rosevere (freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon (https://www.patreon.com/hertzpodcast) and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Episode citation and permanent link Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, May 21) "A GPS in the Garden of Forking Paths (with Amy Orben)", Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/38KPE (https://osf.io/38kpe/) Special Guest: Amy Orben.

By popular demand, Dan and James are kicking it old school and just shooting the breeze. They cover whether scientists should be on Twitter, if Fortnite is ruining our youth, book recommendations, and null oxytocin studies. Stuff they cover and links to obsure references * Should scientists be on twitter? * James runs a Twitter experiment * Scite has now gone live, listen to our episode (https://everythinghertz.com/80) on this platform * Our dreams of a live Hertz episode * Is Fortnite (https://en.wikipedia.org/wiki/Fortnite) killing our youth and the parallels with the “heavy metal” scare * Amy Orben’s screen time study (https://www.nature.com/articles/s41562-018-0506-1) * Multiverse analysis * Book recommendations: Kevin Mitchell’s "Innate" (https://www.amazon.com/Innate-How-Wiring-Brains-Shapes/dp/0691173885), Gareth Leng's "Heart of the brain" (https://mitpress.mit.edu/books/heart-brain) * Daryl dug a hole reference (https://www.youtube.com/watch?v=mDl1i3mpKwM), from the Aussie classic, "The Castle" * A new null oxytocin paper (https://www.sciencedirect.com/science/article/pii/S0306453019300800) and the twitter response, and Dan's response (https://twitter.com/dsquintana/status/1122962998069485573?s=20) * The SANS meeting venue (https://twitter.com/dsquintana/status/1124386664758095877?s=20) * QR codes on posters * The slides to Dan’s oxytocin talk (https://osf.io/q7a4w/) at SANS * The Hertz Hype Cycle * Dan recollects one of the first conversations he had with James Other links - Dan on twitter (www.twitter.com/dsquintana) - James on twitter (www.twitter.com/jamesheathers) - Everything Hertz on twitter (www.twitter.com/hertzpodcast) - Everything Hertz on Facebook (www.facebook.com/everythinghertzpodcast/) Music credits: Lee Rosevere (freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon (https://www.patreon.com/hertzpodcast) and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Episode citation and permanent link Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, May 8) "Back to our dirty unwashed roots", Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/N9BGX (https://osf.io/n9bgx/)

We answer a listener question on the possible negative consequences of the open science movement—are things moving too quickly? Links and things we discuss in the episode: * We have a new logo, if you haven't already noticed... * Contact us via our website form (https://everythinghertz.com/contact)! * Considering the potential downsides of open science * Here come dat boi meme explination (https://knowyourmeme.com/memes/dat-boi) * The dangers of open access by fiat * The role of commercial entities in open science * The “University of Oslo fancy Norway people-pay-taxes oil money bloody library (https://www.ub.uio.no/english/)” * Dropping the success rate of grants to increase the quality of evaluation * Reframing open science reform efforts to a mission of equity and fairness * We don’t know the process behind university sexual harassment/misconduct investigations * Does transparency even matter if people won’t follow up on problems? * James' prediction: If someone starts a journal that ONLY does Registered Report, this will be very successful * The milkshake duck tweet (https://twitter.com/pixelatedboat/status/741904787361300481) and an explanation (https://en.wikipedia.org/wiki/Milkshake_Duck) * Paul Roos and his “no dickheads” policy (https://www.sydneyswans.com.au/news/2011-12-16/the-swan-way) * Linking DOIs * We can't let edge case scenarios, which may not even play out, hobble progress Other links - Dan on twitter (www.twitter.com/dsquintana) - James on twitter (www.twitter.com/jamesheathers) - Everything Hertz on twitter (www.twitter.com/hertzpodcast) - Everything Hertz on Facebook (www.facebook.com/everythinghertzpodcast/) Music credits: Lee Rosevere (freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon (https://www.patreon.com/hertzpodcast) and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Episode citation and permanent link Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, April 15) "More janitors and fewer architects" Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/7ZR9J (https://osf.io/7zr9j/)

We answer our first audio question, on whether academia is too broken to fix, and a second question on whether we’ve ever worried about the possible repercussions of our public critiques and commentary on academia. Show details: Our first audio question is from Erin Williams (@DrErinWill), who asks whether academia is too broken to fix The letter to the editor that got rejected, despite the publication of the response to the letter Harassment in academia Have we ever been worried that someone might say, "I'd never hire those dudes" because of what we say? Other stuff that has happened to us as a result of the podcast Fahrenheit vs. Celsius Supply and demand for academic jobs The criticism that comes with putting yourself out there Links - @ReproRocks (https://twitter.com/ReproRocks): for those working in reproduction to share their work through twitter - The Steven Pinker book - The Better Angels of Our Nature: Why Violence Has Declined (https://www.amazon.com/Better-Angels-Our-Nature-Violence/dp/0143122010) - Twitter thread (https://twitter.com/drderringer/status/1110593951105540096?s=20) from @drderringer - Me too Stem blog (https://metoostem.com/) - Gideon on Twitter: @GidMK (https://twitter.com/GidMK) Other links - Dan on twitter (www.twitter.com/dsquintana) - James on twitter (www.twitter.com/jamesheathers) - Everything Hertz on twitter (www.twitter.com/hertzpodcast) - Everything Hertz on Facebook (www.facebook.com/everythinghertzpodcast/) Music credits: Lee Rosevere (freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon (https://www.patreon.com/hertzpodcast) and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the one dollar tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Episode citation and permanent link Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, April 1) "Too Young To Know, Too Old To Care" Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/W6MER (https://osf.io/345dk/)

We chat with Sean Rife, who the co-founder of scite.ai (https://scite.ai), a start-up that combines natural language processing with a network of experts to evaluate the veracity of scientific work. Here's what we cover and links for a few things we mention * What is scite.ai? * The Winnower (https://thewinnower.com) * Why is there no good (and free) plagiarism detector? * Grobid (https://grobid.readthedocs.io/en/latest/Introduction/) - A machine learning library for extracting, parsing and re-structuring PDFs * Meta-analysis can prop up flawed bodies of literature * The "Too meta (https://xkcd.com/1447/)" XKCD cartoon * What’s the end game for scite? * The 80,000 hours game (https://80000hours.org/articles/can-you-guess/) * Spooner (http://spooner.lyceum.ws), a utility that allows authors of scientific publications to make their work available to the general public (probably) without violating publishing agreements Other links - Sean on twitter (www.twitter.com/seanrife) - Dan on twitter (www.twitter.com/dsquintana) - James on twitter (www.twitter.com/jamesheathers) - Everything Hertz on twitter (www.twitter.com/hertzpodcast) - Everything Hertz on Facebook (www.facebook.com/everythinghertzpodcast/) Music credits: Lee Rosevere (freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon (https://www.patreon.com/hertzpodcast) and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the $1 tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Episode citation and permanent link Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, March 18) "Cites are not endorsements", Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/Q9EYG (https://osf.io/q9eyg/) Special Guest: Sean Rife.

We chat with Henry Drysdale (University of Oxford), co-founder of the COMPare trials project (http://compare-trials.org), which compared clinical trial registrations with reported outcomes in five top medical journals and qualitatively analysed the responses to critical correspondence. Discussion points and links galore: The history behind the COMPare project The two papers that were published: a prospective cohort study (https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-019-3173-2) correcting and monitoring 58 misreported trials and a qualitative analysis (https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-019-3172-3) of researchers’ responses to critical correspondence Ben Goldacre's books (https://www.amazon.co.uk/l/B002C1VRBQ?_encoding=UTF8&redirectedFromKindleDbs=true&rfkd=1&shoppingPortalEnabled=true) What is outcome switching? What were some of the responses to query letters from the authors and journals? Misreporting trials (usually) doesn't lead to patient harm, but it harms the evidence base Where should the buck stop with outcome switching? Would Registered Reports solve this problem? The CONSORT guidelines (http://www.consort-statement.org) Have the journals changed their practices? COMPare on twitter (https://twitter.com/compare_trials) The COMPare website (http://compare-trials.org) Here is Henry on Twitter - @HenryMDrysdale (https://twitter.com/HenryMDrysdale) Here is Ben Goldacre on Twitter - @bengoldacre (https://twitter.com/bengoldacre) Other links - Dan on twitter (www.twitter.com/dsquintana) - James on twitter (www.twitter.com/jamesheathers) - Everything Hertz on twitter (www.twitter.com/hertzpodcast) - Everything Hertz on Facebook (www.facebook.com/everythinghertzpodcast/) Music credits: Lee Rosevere (freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon (https://www.patreon.com/hertzpodcast) and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the $1 tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Episode citation and permanent link Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, March 4) "Clinical trial reporting (with Henry Drysdale)", Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/HBX8R (https://osf.io/hbx8r/) Special Guest: Henry Drysdale.

In this episde, we chat with Lisa DeBruine (University of Glasgow) about her experience with large-scale collaborative science and how her psychology department made the switch from SPSS to R. Discussion points and links galore: Deborah Apthorp's tweet on having to teach SPSS (https://twitter.com/deborahapthorp/status/1092599860212068352), "because that's what students know" People who are involved with teaching R for psychology at the University of Glasgow: @Eavanmac @dalejbarr @McAleerP @clelandwoods @PatersonHelena @emilynordmann Why the #psyTeachR started teaching R for reproducible science Data wrangling vs. statistical analysis The psyTeachR website (https://psyteachr.github.io) Danielle Navarro (https://djnavarro.net), and her R text book (https://learningstatisticswithr.com) that you should read Lisa's "faux" package (https://github.com/debruine/faux) for data simulation Sometimes you can't share data, simulations are a good way around this problem "synthpop" is the name of the package (https://cran.r-project.org/web/packages/synthpop/vignettes/synthpop.pdf) that Dan mentioned that can simulate census data Power analysis can be hard once you go beyond the more conventional statistical tests (e.g., t-tests, ANOVAs etc...) Lisa's OSF page (https://osf.io/4i578/) Dirty code is always better than no code (but the cleaner the better) Live coding is terrifying but a useful teaching tool. Here's Dan live coding how to build a website in R (https://twitter.com/dsquintana/status/1070392412445401088), typos and all Using a Slack group for help The psychological science accelerator (https://psysciacc.org) Chris Chartier (Psych Science Accelerator Director) on Twitter (https://twitter.com/CRChartier) A few of the other (hundreds) of folks involved with the Psych Science Accelerator Director: @PsySciAcc: @CRChartier @BenCJ @JkayFlake @hmoshontz Lisa's Registered Report project (https://osf.io/f7v3n/) on face rating The challenges associated with collaborating with 100+ labs Authorship order Author contributions: CRediT taxonomy (http://dev.biologists.org/content/author-contributions) The DARPA-funding project (https://www.wired.com/story/darpa-wants-to-solve-sciences-replication-crisis-with-robots/) on using AI to determine reproducibility Interacting Minds workshop (http://interactingminds.au.dk/events/single-events/artikel/2-day-workshop-open-science-and-reproducibility/) in Denmark in March on open science and reproducibility Lisa shares what Glasgow is like Lisa has changed her mind about the importance of research metrics (h-index, impact factors etc...) Lisa thinks you should read this paper (https://journals.sagepub.com/doi/abs/10.1177/2515245918770963) on equivalence testing, which includes two former guests, Daniel Lakens (https://everythinghertz.com/guests/daniel-lakens), Anne Scheel (https://everythinghertz.com/guests/anne-scheel), and friend of the show Peder Isager. Here's the latest episode (https://anchor.fm/psychsococlock/episodes/Making-and-breaking-habits---Psych-Soc-OClock---Episode-4-e3327v) from Psych Soc O'Clock Other links - Dan on twitter (www.twitter.com/dsquintana) - James on twitter (www.twitter.com/jamesheathers) - Everything Hertz on twitter (www.twitter.com/hertzpodcast) - Everything Hertz on Facebook (www.facebook.com/everythinghertzpodcast/) Music credits: Lee Rosevere (freemusicarchive.org/music/Lee_Rosevere/) Support us on Patreon (https://www.patreon.com/hertzpodcast) and get bonus stuff! $1 a month or more: Monthly newsletter + Access to behind-the-scenes photos & video via the Patreon app + the the warm feeling you're supporting the show $5 a month or more: All the stuff you get in the first tier PLUS a bonus mini episode every month (extras + the bits we couldn't include in our regular episodes) Episode citation and permanent link Quintana, D.S., Heathers, J.A.J. (Hosts). (2019, February 18) "Large-scale collaborative science (with Lisa DeBruine)", Everything Hertz [Audio podcast], doi: 10.17605/OSF.IO/JDT6F (https://osf.io/jdt6f/) Special Guest: Lisa DeBruine.