PsychDisclosure.org: Grassroots Support for Reforming Reporting Standards in Psychology

There is currently an unprecedented level of doubt regarding the reliability of research findings in psychology. Many recommendations have been made to improve the current situation. In this article, we report results from PsychDisclosure.org, a novel open-science initiative that provides a platform for authors of recently published articles to disclose four methodological design specification details that are not required to be disclosed under current reporting standards but that are critical for accurate interpretation and evaluation of reported findings. Grassroots sentiment—as manifested in the positive and appreciative response to our initiative—indicates that psychologists want to see changes made at the systemic level regarding disclosure of such methodological details. Almost 50% of contacted researchers disclosed the requested design specifications for the four methodological categories (excluded subjects, nonreported conditions and measures, and sample size determination). Disclosed information provided by participating authors also revealed several instances of questionable editorial practices, which need to be thoroughly examined and redressed. On the basis of these results, we argue that the time is now for mandatory methods disclosure statements for all psychology journals, which would be an important step forward in improving the reliability of findings in psychology.

[1]  John C. Norcross,et al.  Ethical Principles of Psychologists and Code of Conduct , 2013 .

[2]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[3]  Brian A. Nosek,et al.  Recommendations for Increasing Replicability in Psychology † , 2013 .

[4]  Preyas S. Desai Editorial - Marketing Science Replication and Disclosure Policy , 2013, Mark. Sci..

[5]  R. Giner-Sorolla,et al.  Science or Art? How Aesthetic Standards Grease the Way Through the Publication Bottleneck but Undermine Science , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[6]  Brian A. Nosek,et al.  An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[7]  J. Ioannidis Why Science Is Not Necessarily Self-Correcting , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[8]  Han L. J. van der Maas,et al.  Science Perspectives on Psychological an Agenda for Purely Confirmatory Research on Behalf Of: Association for Psychological Science , 2022 .

[9]  H. Pashler,et al.  Is the Replicability Crisis Overblown? Three Arguments Examined , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[10]  S. Fiedler,et al.  Psychologists Are Open to Change, yet Wary of Rules , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[11]  Leif D. Nelson,et al.  A 21 Word Solution , 2012 .

[12]  Jeffrey R. Spies,et al.  Scientific Utopia: II. Restructuring incentives and practices to promote truth over publishability , 2012, 1205.4251.

[13]  G. Loewenstein,et al.  Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling , 2012, Psychological science.

[14]  W. Levelt,et al.  Flawed science: The fraudulent research practices of social psychologist Diederik Stapel , 2012 .

[15]  Etienne P LeBel,et al.  Fearing the Future of Empirical Psychology: Bem's (2011) Evidence of Psi as a Case Study of Deficiencies in Modal Research Practice , 2011, Review of General Psychology.

[16]  Peter DeScioli,et al.  The Omission Strategy , 2011, Psychological science.

[17]  J. Pennebaker,et al.  Are Women Really More Talkative Than Men? , 2007, Science.

[18]  J. Platt Strong Inference , 2007 .

[19]  L. HARKing: Hypothesizing After the Results are Known , 2002 .

[20]  P. Meehl Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. , 1978 .

[21]  P. Armitage,et al.  Repeated Significance Tests on Accumulating Data , 1969 .

[22]  P. Meehl Theory-Testing in Psychology and Physics: A Methodological Paradox , 1967, Philosophy of Science.

[23]  L. Festinger,et al.  Cognitive consequences of forced compliance. , 2011, Journal of abnormal psychology.