Reporting and Interpreting Research in PSPB: Practices, Principles, and Pragmatics

This article is designed to provide psychologists who publish articles in Personality and Social Psychology Bulletin (PSPB) with a set of basic issues to consider when reporting their analyses and results. We first assessed the current reporting practices of social and personality psychologists by conducting an analysis of PSPB articles published in the first half of 2007. We evaluated the completeness of these reports with respect to the level of detail in both the Method and Results sections. We then used this information to develop recommendations that we hope will enhance the reporting of quantitative research in social and personality psychology. These suggestions emphasize ways to increase transparency in research reports. Transparency facilitates replication and a critical evaluation of research, thereby promoting scientific progress.

[1]  Jeremy Freese,et al.  Replication Standards for Quantitative Social Science , 2007 .

[2]  Matthew S. Fritz,et al.  Mediation analysis. , 2019, Annual review of psychology.

[3]  R. Vandenberg,et al.  A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research , 2000 .

[4]  Robert C. MacCallum,et al.  SPECIFICATION SEARCHES IN COVARIANCE STRUCTURE MODELING , 1986 .

[5]  P. Barrett.  Structural equation modelling : Adjudging model fit , 2007 .

[6]  Kristopher J Preacher,et al.  SPSS and SAS procedures for estimating indirect effects in simple mediation models , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[7]  Phillip R. Shaver,et al.  Attachment dimensions and the big five personality traits: Associations and comparative ability to predict relationship quality , 2006 .

[8]  Stephen G. West,et al.  Publication Trends in JPSP: Stability and Change in Topics, Methods, and Theories Across Two Decades , 1992 .

[9]  S. West,et al.  A comparison of methods to test mediation and other intervening variable effects. , 2002, Psychological methods.

[10]  G. Cumming,et al.  Statistical Reform in Psychology , 2007, Psychological science.

[11]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[12]  Richard J. Harris.  Significance Tests Have Their Place , 1997 .

[13]  Andrew F. Hayes,et al.  The Sage sourcebook of advanced data analysis methods for communication research , 2008 .

[14]  T. Little,et al.  To Parcel or Not to Parcel: Exploring the Question, Weighing the Merits , 2002 .

[15]  Yuk Fai Cheong,et al.  HLM 6: Hierarchical Linear and Nonlinear Modeling , 2000 .

[16]  Craig K. Enders,et al.  Centering predictor variables in cross-sectional multilevel models: a new look at an old issue. , 2007, Psychological methods.

[17]  Rick H. Hoyle,et al.  Determining the Number of Factors in Exploratory and Confirmatory Factor Analysis , 2004 .

[18]  Rex B. Kline,et al.  Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research , 2004 .

[19]  Duane T. Wegener,et al.  Evaluating the use of exploratory factor analysis in psychological research. , 1999 .

[20]  D. Wegener,et al.  The problem of equivalent models in applications of covariance structure analysis. , 1993, Psychological bulletin.

[21]  B. Thompson,et al.  Use of Structure Coefficients in Published Multiple Regression Articles: β is not Enough , 2001 .

[22]  Barbara G. Tabachnick,et al.  Computer-Assisted Research Design and Analysis , 2000 .

[23]  D. Russell.  In Search of Underlying Dimensions: The Use (and Abuse) of Factor Analysis in Personality and Social Psychology Bulletin , 2002 .

[24]  S. Maxwell,et al.  Bivariate median splits and spurious statistical significance. , 1993 .

[25]  Lawrence C. Hamilton,et al.  V. CONTEMPORARY ADVANCES AND CLASSIC ADVICE FOR ANALYZING MEDIATING AND MODERATING VARIABLES , 2006 .

[26]  Norman Cliff,et al.  The eigenvalues-greater-than-one rule and the reliability of components. , 1988 .

[27]  Kristopher J Preacher,et al.  Use of the extreme groups approach: a critical reexamination and new recommendations. , 2005, Psychological methods.

[28]  G. Cumming,et al.  Inference by eye: confidence intervals and how to read pictures of data. , 2005, The American psychologist.

[29]  Leland Wilkinson,et al.  Statistical Methods in Psychology Journals Guidelines and Explanations , 2005 .

[30]  M. Zanna,et al.  Establishing a causal chain: why experiments are often more effective than mediational analyses in examining psychological processes. , 2005, Journal of personality and social psychology.

[31]  Ralph L. Rosnow,et al.  Essentials of Behavioral Research: Methods and Data Analysis , 1984 .

[32]  Patrick E. McKnight.  Missing Data: A Gentle Introduction , 2007 .

[33]  D. A. Kenny,et al.  The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. , 1986, Journal of personality and social psychology.

[34]  R. P. McDonald,et al.  Principles and practice in reporting structural equation analyses. , 2002, Psychological methods.

[35]  R. Dodhia.  A Review of Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences (3rd ed.) , 2005 .

[36]  C. Judd,et al.  When moderation is mediated and mediation is moderated. , 2005, Journal of personality and social psychology.

[37]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[38]  P. Shrout,et al.  Mediation in experimental and nonexperimental studies: new procedures and recommendations. , 2002, Psychological methods.

[39]  K. Widaman.  Common Factors Versus Components: Principals and Principles, Errors and Misconceptions , 2007 .

[40]  D. Mackinnon.  Introduction to Statistical Mediation Analysis , 2008 .

[41]  Andrew F. Hayes,et al.  Contemporary approaches to assessing mediation in communication research. , 2008 .

[42]  P. Fayers.  Item Response Theory for Psychologists , 2004, Quality of Life Research.

[43]  B P O'Connor,et al.  SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test , 2000, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[44]  J. Edwards,et al.  Methods for integrating moderation and mediation: a general analytical framework using moderated path analysis. , 2007, Psychological methods.

[45]  H. Marsh,et al.  In Search of Golden Rules: Comment on Hypothesis-Testing Approaches to Setting Cutoff Values for Fit Indexes and Dangers in Overgeneralizing Hu and Bentler's (1999) Findings , 2004 .

[46]  I G Kreft,et al.  The Effect of Different Forms of Centering in Hierarchical Linear Models. , 1995, Multivariate behavioral research.

[47]  Michael J. Marks,et al.  The null hypothesis significance-testing debate and its implications for personality research. , 2007 .

[48]  D. Borsboom,et al.  The poor availability of psychological research data for reanalysis. , 2006, The American psychologist.

[49]  J. H. Steiger.  Structural Model Evaluation and Modification: An Interval Estimation Approach. , 1990, Multivariate behavioral research.

[50]  J. G. Adair,et al.  The explosion of knowledge, references, and citations. Psychology's unique response to a crisis. , 2003, The American psychologist.

[51]  Amy M. Buddie,et al.  Twenty Years of PSPB: Trends in Content, Design, and Analysis , 1999 .

[52]  Kristopher J Preacher,et al.  On the practice of dichotomization of quantitative variables. , 2002, Psychological methods.

[53]  Larry E. Toothaker,et al.  Multiple Regression: Testing and Interpreting Interactions , 1991 .

[54]  James C. Anderson,et al.  STRUCTURAL EQUATION MODELING IN PRACTICE: A REVIEW AND RECOMMENDED TWO-STEP APPROACH , 1988 .

[55]  H. Stern,et al.  The use of multiple imputation for the analysis of missing data. , 2001, Psychological methods.

[56]  J. Schafer,et al.  Missing data: our view of the state of the art. , 2002, Psychological methods.

[57]  Harry T. Reis,et al.  Publication Trends in JPSP: A Three-Decade Review , 1992 .