The meaning of "significance" for different types of research [translated and annotated by Eric-Jan Wagenmakers, Denny Borsboom, Josine Verhagen, Rogier Kievit, Marjan Bakker, Angelique Cramer, Dora Matzke, Don Mellenbergh, and Han L. J. van der Maas]. 1969.

Adrianus Dingeman de Groot (1914-2006) was one of the most influential Dutch psychologists. He became famous for his work "Thought and Choice in Chess", but his main contribution was methodological--De Groot co-founded the Department of Psychological Methods at the University of Amsterdam (together with R. F. van Naerssen), founded one of the leading testing and assessment companies (CITO), and wrote the monograph "Methodology" that centers on the empirical-scientific cycle: observation-induction-deduction-testing-evaluation. Here we translate one of De Groot's early articles, published in 1956 in the Dutch journal Nederlands Tijdschrift voor de Psychologie en Haar Grensgebieden. This article is more topical now than it was almost 60years ago. De Groot stresses the difference between exploratory and confirmatory ("hypothesis testing") research and argues that statistical inference is only sensible for the latter: "One 'is allowed' to apply statistical tests in exploratory research, just as long as one realizes that they do not have evidential impact". De Groot may have also been one of the first psychologists to argue explicitly for preregistration of experiments and the associated plan of statistical analysis. The appendix provides annotations that connect De Groot's arguments to the current-day debate on transparency and reproducibility in psychological science.

[1]  Joshua Carp,et al.  The secret lives of experiments: Methods reporting in the fMRI literature , 2012, NeuroImage.

[2]  John Monks,et al.  Bad science. , 1997, Nursing standard (Royal College of Nursing (Great Britain) : 1987).

[3]  Neuroskeptic The Nine Circles of Scientific Hell , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[4]  Han L. J. van der Maas,et al.  Science Perspectives on Psychological an Agenda for Purely Confirmatory Research on Behalf Of: Association for Psychological Science , 2022 .

[5]  G. Loewenstein,et al.  Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling , 2012, Psychological science.

[6]  W. K. Simmons,et al.  Circular analysis in systems neuroscience: the dangers of double dipping , 2009, Nature Neuroscience.

[7]  Stuart J. Ritchie,et al.  Failing the Future: Three Unsuccessful Attempts to Replicate Bem's ‘Retroactive Facilitation of Recall’ Effect , 2012, PloS one.

[8]  R. MacCallum,et al.  Model modifications in covariance structure analysis: the problem of capitalization on chance. , 1992, Psychological bulletin.

[9]  Ben Goldacre,et al.  Bad Pharma: How Drug Companies Mislead Doctors and Harm Patients , 2012 .

[10]  D. Bem Feeling the future: experimental evidence for anomalous retroactive influences on cognition and affect. , 2011, Journal of personality and social psychology.

[11]  Leif D. Nelson,et al.  Correcting the past: failures to replicate ψ. , 2012, Journal of personality and social psychology.

[12]  Z. Dienes Bayesian Versus Orthodox Statistics: Which Side Are You On? , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[13]  James G. Scott,et al.  Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem , 2010, 1011.2333.

[14]  E. Wagenmakers,et al.  Why psychologists must change the way they analyze their data: the case of psi: comment on Bem (2011). , 2011, Journal of personality and social psychology.

[15]  Meredith Wadman,et al.  NIH mulls rules for validating key results , 2013, Nature.

[16]  W. Rozeboom,et al.  Methodology: Foundations of Inference and Research in the Behavioral Sciences. , 1971 .

[17]  Brian A. Nosek,et al.  Scientific Utopia: I. Opening Scientific Communication , 2012, ArXiv.

[18]  C. Chambers Registered Reports: A new publishing initiative at Cortex , 2013, Cortex.

[19]  N. Kerr HARKing: Hypothesizing After the Results are Known , 1998, Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc.

[20]  H. Pashler,et al.  Editors’ Introduction to the Special Section on Replicability in Psychological Science , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[21]  J. Brooks Why most published research findings are false: Ioannidis JP, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece , 2008 .

[22]  Leif D. Nelson,et al.  False-Positive Psychology , 2011, Psychological science.

[23]  P. L. Wittchen APPENDIX B. REFERENCES , 2006 .

[24]  Registered Reports and Replications in Attention, Perception, & Psychophysics , 2013 .

[25]  Elliot A. Ludvig,et al.  Comparative psychology and the grand challenge of drug discovery in psychiatry and neurodegeneration , 2012, Behavioural Processes.

[26]  M. Stephens,et al.  Bayesian statistical methods for genetic association studies , 2009, Nature Reviews Genetics.

[27]  F. Prinz,et al.  Believe it or not: how much can we rely on published data on potential drug targets? , 2011, Nature Reviews Drug Discovery.

[28]  Brian A. Nosek,et al.  Scientific Utopia , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[29]  I. Reid Bad Pharma: How Drug Companies Mislead Doctors and Harm Patients , 2013, British Journal of Psychiatry.

[30]  H. Pashler,et al.  Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition 1 , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[31]  C. Glenn Begley,et al.  Raise standards for preclinical cancer research , 2012 .