The poor availability of syntaxes of structural equation modeling

ABSTRACT The syntax or codes used to fit Structural Equation Models (SEMs) convey valuable information on model specifications and the manner in which SEMs are estimated. We requested SEM syntaxes from a random sample of 229 articles (published in 1998–2013) that ran SEMs using LISREL, AMOS, or Mplus. After exchanging over 500 emails, we ended up obtaining a meagre 57 syntaxes used in these articles (24.9% of syntaxes we requested). Results considering the 129 (corresponding) authors who replied to our request showed that the odds of the syntax being lost increased by 21% per year passed since publication of the article, while the odds of actually obtaining a syntax dropped by 13% per year. So SEM syntaxes that are crucial for reproducibility and for correcting errors in the running and reporting of SEMs are often unavailable and get lost rapidly. The preferred solution is mandatory sharing of SEM syntaxes alongside articles or in data repositories.

[1]  Yolanda Gil,et al.  Enhancing reproducibility for computational methods , 2016, Science.

[2]  J. Ioannidis,et al.  Public Availability of Published Research Data in High-Impact Journals , 2011, PloS one.

[3]  James Howison,et al.  Software in the scientific literature: Problems with seeing, finding, and using software mentioned in the biology literature , 2016, J. Assoc. Inf. Sci. Technol..

[4]  Yo In'nami,et al.  Can Structural Equation Models in Second Language Testing and Learning Research be Successfully Replicated? , 2010 .

[5]  Florence Debarre,et al.  The Availability of Research Data Declines Rapidly with Article Age , 2013, Current Biology.

[6]  R. Purc-Stephenson,et al.  Reporting practices in confirmatory factor analysis: an overview and some recommendations. , 2009, Psychological methods.

[7]  W. Vanpaemel,et al.  Are We Wasting a Good Crisis? The Availability of Psychological Research Data after the Storm , 2015 .

[8]  H. Van de Sompel,et al.  Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot , 2014, PloS one.

[9]  James R. Craig,et al.  Retention of raw data: A problem revisited. , 1973 .

[10]  D. Borsboom,et al.  The poor availability of psychological research data for reanalysis. , 2006, The American psychologist.

[11]  R. MacCallum,et al.  Applications of structural equation modeling in psychological research. , 2000, Annual review of psychology.

[12]  V. Stodden,et al.  Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals , 2013, PloS one.

[13]  Rene Weber,et al.  Undisclosed Flexibility in Computing and Reporting Structural Equation Models in Communication Science , 2015 .

[14]  Yo In’nami,et al.  Structural Equation Modeling in Language Testing and Learning Research: A Review , 2011 .

[15]  G. Cumming,et al.  The influence of journal submission guidelines on authors' reporting of statistics and use of open research practices , 2017, PloS one.

[16]  Michèle B. Nuijten,et al.  Journal Data Sharing Policies and Statistical Reporting Inconsistencies in Psychology , 2017 .

[17]  Diomidis Spinellis,et al.  The decay and failures of web references , 2003, CACM.

[18]  Susann Fiedler,et al.  Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency , 2016, PLoS biology.

[19]  Ernesto Reuben,et al.  (Un)Available upon Request: Field Experiment on Researchers' Willingness to Share Supplementary Materials , 2012, Accountability in research.

[20]  Conor V Dolan,et al.  Stereotype threat and group differences in test performance: a question of measurement invariance. , 2005, Journal of personality and social psychology.

[21]  Brian A. Nosek,et al.  Scientific Utopia , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[22]  J. Wicherts,et al.  Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results , 2011, PloS one.

[23]  B. T. Sampath Kumar,et al.  HTTP 404-page (not) found: Recovery of decayed URL citations , 2013, J. Informetrics.

[24]  John Fox,et al.  OpenMx: An Open Source Extended Structural Equation Modeling Framework , 2011, Psychometrika.

[25]  Michèle B. Nuijten,et al.  The prevalence of statistical reporting errors in psychology (1985–2013) , 2015, Behavior Research Methods.

[26]  Haywantee Ramkissoon,et al.  Use of Structural Equation Modeling in Tourism Research , 2013 .

[27]  Leroy Wolins,et al.  Responsibility for Raw Data. , 1962 .

[28]  Rachna Shah,et al.  Use of structural equation modeling in operations management research: Looking back and forward ☆ , 2006 .

[29]  Jeffrey R. Spies.  The Open Science Framework: Improving Science by Making It Open and Accessible , 2013 .

[30]  Yves Rosseel,et al.  lavaan: An R Package for Structural Equation Modeling , 2012 .