Predictive model selection in partial least squares path modeling (PLS-PM)

Predictive model selection metrics are used to select models with the highest out-of-sample predictive power among a set of models. R2 and related metrics, which are heavily used in partial least squares path modeling, are often mistaken as predictive metrics. We introduce information theoretic model selection criteria that are designed for out-of-sample prediction and which do not require creating a holdout sample. Using a Monte Carlo study, we compare the performance of frequently used model evaluation criteria and information theoretic criteria in selecting the best predictive model under various conditions of sample size, effect size, loading patterns, and data distribution.

[1]  D. Straub,et al.  Editor's comments: a critical look at the use of PLS-SEM in MIS quarterly , 2012 .

[2]  G. Celeux,et al.  An entropy criterion for assessing the number of clusters in a mixture model , 1996 .

[3]  P M Bentler,et al.  Choice of structural model via parsimony: a rationale based on precision. , 1989, Psychological bulletin.

[4]  Marko Sarstedt,et al.  The Use of Partial Least Squares Structural Equation Modeling in Strategic Management Research: A Review of Past Practices and Recommendations for Future Applications , 2012 .

[5]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[6]  Marko Sarstedt,et al.  An assessment of the use of partial least squares structural equation modeling in marketing research , 2012 .

[7]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[8]  Christopher Hitchcock,et al.  Prediction Versus Accommodation and the Risk of Overfitting , 2004, The British Journal for the Philosophy of Science.

[9]  A. McQuarrie,et al.  Regression and Time Series Model Selection , 1998 .

[10]  Mary Tate,et al.  Comparing Out-of-Sample Predictive Ability of PLS, Covariance, and Regression Models , 2014, ICIS.

[11]  Arun Rai,et al.  Predictive Validity and Formative Measurement in Structural Equation Modeling: Embracing Practical Relevance , 2013, ICIS.

[12]  D. M. Allen,et al.  Determining the number of components in mixtures of linear models , 2001 .

[13]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[14]  Chih-Ling Tsai,et al.  Regression model selection—a residual likelihood approach , 2002 .

[15]  Herman Wold,et al.  Model Construction and Evaluation When Theoretical Knowledge Is Scarce , 1980 .

[16]  Malcolm R. Forster,et al.  How to Tell When Simpler, More Unified, or Less Ad Hoc Theories will Provide More Accurate Predictions , 1994, The British Journal for the Philosophy of Science.

[17]  Marko Sarstedt,et al.  How collinearity affects mixture regression results , 2015 .

[18]  M. Stone.  An Asymptotic Equivalence of Choice of Model by Cross‐Validation and Akaike's Criterion , 1977 .

[19]  Stacie Petter,et al.  On the use of partial least squares path modeling in accounting research , 2011, Int. J. Account. Inf. Syst..

[20]  Rick L. Andrews,et al.  A Comparison of Segment Retention Criteria for Finite Mixture Logit Models , 2003 .

[21]  I. J. Myung,et al.  The Importance of Complexity in Model Selection. , 2000, Journal of mathematical psychology.

[22]  J. Shao.  Linear Model Selection by Cross-validation , 1993 .

[23]  Rick L. Andrews,et al.  Retention of latent segments in regression-based marketing models , 2003 .

[24]  H. Wold.  Causal flows with latent variables: Partings of the ways in the light of NIPALS modelling , 1974 .

[25]  Gérard Govaert,et al.  Assessing a Mixture Model for Clustering with the Integrated Completed Likelihood , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Galit Shmueli,et al.  Predictive Analytics in Information Systems Research , 2010, MIS Q..

[27]  Shalabh.  Statistical Learning from a Regression Perspective , 2009 .