Parsimony in model selection: Tools for assessing fit propensity.

Theories can be represented as statistical models for empirical testing. There is a vast literature on model selection and multimodel inference that focuses on how to assess which statistical model, and therefore which theory, best fits the available data. For example, given some data, one can compare models on various information criterion or other fit statistics. However, what these indices fail to capture is the full range of counterfactuals. That is, some models may fit the given data better not because they represent a more correct theory, but simply because these models have more fit propensity-a tendency to fit a wider range of data, even nonsensical data, better. Current approaches fall short in considering the principle of parsimony (Occam's Razor), often equating it with the number of model parameters. Here we offer a toolkit for researchers to better study and understand parsimony through the fit propensity of structural equation models. We provide an R package (ockhamSEM) built on the popular lavaan package. To illustrate the importance of evaluating fit propensity, we use ockhamSEM to investigate the factor structure of the Rosenberg Self-Esteem Scale. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

[1]  Carl F. Falk,et al.  Recovering Substantive Factor Loadings in the Presence of Acquiescence Bias: A Comparison of Three Approaches , 2014, Multivariate behavioral research.

[2]  M. Rosenberg.  Society and the adolescent self-image , 1966 .

[3]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[4]  Mark A. Pitt,et al.  Model Evaluation, Testing and Selection , 2005 .

[5]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[6]  H. Joe.  Generating random correlation matrices based on partial correlations , 2006 .

[7]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[8]  Julia Kastner,et al.  Introduction to Robust Estimation and Hypothesis Testing , 2005 .

[9]  Timothy R. Brick,et al.  OpenMx 2.0: Extended Structural Equation and Statistical Modeling , 2015, Psychometrika.

[10]  S. Reise,et al.  Is the Bifactor Model a Better Model or Is It Just Better at Modeling Implausible Responses? Application of Iteratively Reweighted Least Squares to the Rosenberg Self-Esteem Scale , 2016, Multivariate behavioral research.

[11]  Yves Rosseel,et al.  lavaan: An R Package for Structural Equation Modeling , 2012 .

[12]  M. Browne,et al.  Cross-Validation Of Covariance Structures. , 1983, Multivariate behavioral research.

[13]  Steven P. Reise,et al.  The role of the bifactor model in resolving dimensionality issues in health outcomes measures , 2007, Quality of Life Research.

[14]  M. Peruggia.  Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach (2nd ed.) , 2003 .

[15]  P A Kolers,et al.  A problem for theory. , 1972, Vision research.

[16]  Mark A. Pitt,et al.  Toward a method of selecting among computational models of cognition. , 2002 .

[17]  Kristopher J Preacher,et al.  The Role of Model Complexity in the Evaluation of Structural Equation Models , 2003 .

[18]  D. J. Lee.  Society and the Adolescent Self-Image , 1969 .

[19]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[20]  Bin Yu,et al.  Model Selection and the Principle of Minimum Description Length , 2001 .

[21]  I. J. Myung,et al.  Toward a method of selecting among computational models of cognition. , 2002, Psychological review.

[22]  Peter M. Bentler,et al.  EQS : structural equations program manual , 1989 .

[23]  Tiffany A. Whittaker,et al.  Examining the Factor Structure of the Self-Compassion Scale in Four Distinct Populations: Is the Use of a Total Scale Score Justified? , 2017, Journal of personality assessment.

[24]  Kenneth A. Bollen,et al.  Representing general theoretical concepts in structural equation models: the role of composite variables , 2008, Environmental and Ecological Statistics.

[25]  J. H. Steiger.  Statistically based tests for the number of common factors , 1980 .

[26]  Kristopher J Preacher,et al.  Quantifying Parsimony in Structural Equation Modeling , 2006, Multivariate behavioral research.

[27]  N. Cliff.  Answering Ordinal Questions with Ordinal Data Using Ordinal Statistics. , 1996, Multivariate behavioral research.

[28]  Albert Satorra,et al.  Testing model nesting and equivalence. , 2008, Psychological methods.

[29]  Wes Bonifay,et al.  On the Complexity of Item Response Theory Models , 2017, Multivariate behavioral research.

[30]  S. Reise,et al.  Three Concerns With Applying a Bifactor Model as a Structure of Psychopathology , 2017 .

[31]  J. Pearl,et al.  EIGHT MYTHS ABOUT CAUSALITY AND STRUCTURAL EQUATION MODELS , 2013 .

[32]  Stan Lipovetsky,et al.  Generalized Latent Variable Modeling: Multilevel,Longitudinal, and Structural Equation Models , 2005, Technometrics.

[33]  Samuel B. Green,et al.  Graphical Displays for Understanding SEM Model Similarity , 2017 .

[34]  Los Angeles,et al.  An Integrative Framework of Model Evaluation , 2015 .

[35]  Dorota Kurowicka,et al.  Generating random correlation matrices based on vines and extended onion method , 2009, J. Multivar. Anal..

[36]  M. Donnellan,et al.  Extending Structural Analyses of the Rosenberg Self-Esteem Scale to Consider Criterion-Related Validity: Can Composite Self-Esteem Scores Be Good Enough? , 2016, Journal of personality assessment.

[37]  Herbert W. Marsh,et al.  Goodness of fit in confirmatory factor analysis: The effects of sample size and model parsimony , 1994 .

[38]  L. Tucker,et al.  A reliability coefficient for maximum likelihood factor analysis , 1973 .

[39]  Judea Pearl,et al.  A Theory of Inferred Causation , 1991, KR.

[40]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[41]  S. Reise,et al.  Scoring and Modeling Psychological Measures in the Presence of Multidimensionality , 2013, Journal of personality assessment.