Research and Educational Practice Improving the Dependability of Research in Personality and Social Psychology : Recommendations for

In this article, the Society for Personality and Social Psychology (SPSP) Task Force on Publication and Research Practices offers a brief statistical primer and recommendations for improving the dependability of research. Recommendations for research practice include (a) describing and addressing the choice of N (sample size) and consequent issues of statistical power, (b) reporting effect sizes and 95% confidence intervals (CIs), (c) avoiding “questionable research practices” that can inflate the probability of Type I error, (d) making available research materials necessary to replicate reported results, (e) adhering to SPSP’s data sharing policy, (f) encouraging publication of high-quality replication studies, and (g) maintaining flexibility and openness to alternative standards and methods. Recommendations for educational practice include (a) encouraging a culture of “getting it right,” (b) teaching and encouraging transparency of data reporting, (c) improving methodological instruction, and (d) modeling sound science and supporting junior researchers who seek to “get it right.”

[1]  Ken Kelley,et al.  Sample Size Planning , 2010, The SAGE Encyclopedia of Research Design.

[2]  Brian A. Nosek,et al.  Recommendations for Increasing Replicability in Psychology † , 2013 .

[3]  J. Rossi.  Statistical Power Analysis , 2012 .

[4]  U. Schimmack.  The ironic effect of significant results on the credibility of multiple-study articles. , 2012, Psychological methods.

[5]  E. Masicampo,et al.  A peculiar prevalence of p values just below .05 , 2012, Quarterly journal of experimental psychology.

[6]  E. Yong.  Replication studies: Bad copy , 2012, Nature.

[7]  G. Loewenstein,et al.  Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling , 2012, Psychological science.

[8]  P. Diaconis.  Theories of Data Analysis: From Magical Thinking Through Classical Statistics , 2011 .

[9]  G. Cumming.  Understanding the New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analysis , 2011 .

[10]  Nigel Dallow,et al.  The perils with the misuse of predictive power. , 2011, Pharmaceutical statistics.

[11]  D. Bem.  Feeling the future: experimental evidence for anomalous retroactive influences on cognition and affect. , 2011, Journal of personality and social psychology.

[12]  C. Chiu,et al.  Mapping the Structure and Dynamics of Psychological Knowledge: Forty Years of APA Journal Citations (1970–2009) , 2009 .

[13]  H. Pashler,et al.  Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition 1 , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[14]  J. Brooks.  Why most published research findings are false: Ioannidis JP, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece , 2008 .

[15]  Ken Kelley,et al.  Methods for the Behavioral, Educational, and Social Sciences: An R package , 2007, Behavior research methods.

[16]  Edgar Erdfelder,et al.  G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences , 2007, Behavior research methods.

[17]  Michael J. Marks,et al.  The null hypothesis significance-testing debate and its implications for personality research. , 2007 .

[18]  Russell V. Lenth,et al.  Post Hoc Power : Tables and Commentary , 2007 .

[19]  R. McGrath,et al.  When effect sizes disagree: the case of r and d. , 2006, Psychological methods.

[20]  R. P. Carver.  The Case Against Statistical Significance Testing 1 , 2006 .

[21]  Scott E. Maxwell,et al.  On the Post Hoc Power in Testing Mean Differences , 2005 .

[22]  G. Cumming,et al.  Inference by eye: confidence intervals and how to read pictures of data. , 2005, The American psychologist.

[23]  C. F. Bond,et al.  One Hundred Years of Social Psychology Quantitatively Described , 2003 .

[24]  B. Muthén,et al.  How to Use a Monte Carlo Study to Decide on Sample Size and Determine Power , 2002 .

[25]  Russell V. Lenth,et al.  Some Practical Guidelines for Effective Sample Size Determination , 2001 .

[26]  D. Heisey,et al.  The Abuse of Power , 2001 .

[27]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[28]  Ronald C. Serlin,et al.  Equivalence confidence intervals for two-group comparisons of means , 1998 .

[29]  Michael L. Dennis,et al.  Practical power analysis for substance abuse health services research. , 1997 .

[30]  Jacob Cohen.  The earth is round (p < .05) , 1994 .

[31]  Gwenn W. Gröndal,et al.  Meta-analytic procedures for social research , 1993 .

[32]  J. L. Rogers,et al.  Using significance tests to evaluate equivalence between two experimental groups. , 1993, Psychological bulletin.

[33]  H. Chaiklin,et al.  Methodology and Epistemology for Social Science: Selected Papers , 1990 .

[34]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[35]  Robert P. Abelson,et al.  A Variance Explanation Paradox : When a Little is a Lot , 1985 .

[36]  P. Meehl.  Theory-Testing in Psychology and Physics: A Methodological Paradox , 1967, Philosophy of Science.