An Adaptive Bayesian Lasso Approach with Spike-and-Slab Priors to Identify Multiple Linear and Nonlinear Effects in Structural Equation Models

In applied research, such as with motivation theories, typically many variables are theoretically implied predictors of an outcome and several interactions are assumed (e.g., Watt, 2004). However, estimation problems that might arise when several interaction and/or quadratic effects are analyzed simultaneously have not been investigated because simulation studies on interaction effects in the structural equation modeling framework have mainly focused on small models that contain single interaction effects. In this article, we show that traditional approaches can provide estimates with low accuracy when complex models are estimated. We introduce an adaptive Bayesian lasso approach with spike-and-slab priors that overcomes this problem. Using a complex model in a simulation study, we show that the parameter estimates of the proposed approach are more accurate in situations with high multicollinearity or low reliability compared with a standard Bayesian lasso approach and typical frequentist approaches (i.e., unconstrained product indicator approach and latent moderated structures approach).

[1]  Fan Yang,et al.  Nonlinear structural equation models: The Kenny-Judd model with Interaction effects , 1996 .

[2]  J. Griffin,et al.  Inference with normal-gamma prior distributions in regression problems , 2010 .

[3]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[4]  David B. Dunson,et al.  Generalized Beta Mixtures of Gaussians , 2011, NIPS.

[5]  G. Casella Empirical Bayes Gibbs sampling. , 2001, Biostatistics.

[6]  Ludwig Fahrmeir,et al.  Bayesian regularisation in structured additive regression: a unifying perspective on shrinkage, smoothing and predictor selection , 2010, Stat. Comput..

[7]  Stephen G West,et al.  Estimating Latent Variable Interactions With Nonnormal Observed Data: A Comparison of Four Approaches , 2012, Multivariate behavioral research.

[8]  Honghong Xu,et al.  Relationship between resilience, stress and burnout among civil servants in Beijing, China: Mediating and moderating effect analysis , 2015 .

[9]  Jeffrey S. Rosenthal,et al.  Optimal Proposal Distributions and Adaptive MCMC , 2011 .

[10]  B. Muthén,et al.  Growth mixture modeling , 2008 .

[11]  H. White Maximum Likelihood Estimation of Misspecified Models , 1982 .

[12]  Aki Vehtari,et al.  Sparsity information and regularization in the horseshoe and other shrinkage priors , 2017, 1707.01694.

[13]  J. Eccles Expectancies, values and academic behaviors , 1983 .

[14]  Xinyuan Song,et al.  Bayesian Adaptive Lasso for Ordinal Regression With Latent Variables , 2017 .

[15]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[16]  E. George,et al.  Journal of the American Statistical Association is currently published by American Statistical Association. , 2007 .

[17]  Jaeyong Lee,et al.  GENERALIZED DOUBLE PARETO SHRINKAGE. , 2011, Statistica Sinica.

[18]  Allan Wigfield,et al.  Students' achievement values, goal orientations, and interest: Definitions, development, and relations to achievement outcomes , 2010 .

[19]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[20]  H. Moosbrugger,et al.  On the Performance of Likelihood-Based Difference Tests in Nonlinear Structural Equation Models , 2015 .

[21]  W. Chaplin,et al.  The next generation of moderator research in personality psychology. , 1991, Journal of personality.

[22]  W. Chaplin Moderator and mediator models in personality research: A basic introduction. , 2007 .

[23]  A. Gelman Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper) , 2004 .

[24]  L. Hawk,et al.  Internalizing and externalizing problem behavior and early adolescent substance use: a test of a latent variable interaction and conditional indirect effects. , 2014, Psychology of addictive behaviors : journal of the Society of Psychologists in Addictive Behaviors.

[25]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[26]  G. Riva,et al.  Attachment insecurities, maladaptive perfectionism, and eating disorder symptoms: A latent mediated and moderated structural equation modeling analysis across diagnostic groups , 2014, Psychiatry Research.

[27]  D. Hibbs On analyzing the effects of policy interventions : Box-Jenkins and Box-Tiao vs. structural equation models , 1977 .

[28]  Edward H. Ip,et al.  A Bayesian Modeling Approach for Generalized Semiparametric Structural Equation Models , 2013, Psychometrika.

[29]  George A. Marcoulides,et al.  Interaction and Nonlinear Effects in Structural Equation Modeling , 1998 .

[30]  Bengt Muthén,et al.  Dynamic Latent Class Analysis , 2017 .

[31]  Chris Hans Bayesian lasso regression , 2009 .

[32]  Sara van de Geer,et al.  The group Lasso , 2011 .

[33]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[34]  John J. McArdle,et al.  Regularized Structural Equation Modeling , 2015, Multivariate behavioral research.

[35]  Xin-Yuan Song,et al.  Bayesian Regularized Multivariate Generalized Latent Variable Models , 2017 .

[36]  M. Betancourt,et al.  Hamiltonian Monte Carlo for Hierarchical Models , 2013, 1312.0906.

[37]  Ioannis Ntzoufras,et al.  On Bayesian lasso variable selection and the specification of the shrinkage parameter , 2012, Stat. Comput..

[38]  Allan Wigfield,et al.  Expectancy-value theory: retrospective and prospective , 2010 .

[39]  Radford M. Neal MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[40]  Holger Brandt,et al.  A general non-linear multilevel structural equation mixture model , 2014, Front. Psychol..

[41]  H. Zou The Adaptive Lasso and Its Oracle Properties , 2006 .

[42]  Daniel J Bauer,et al.  A More General Model for Testing Measurement Invariance and Differential Item Functioning , 2017, Psychological methods.

[43]  Alexander Seeshing Yeung,et al.  Expectancy-value in mathematics, gender and socioeconomic background as predictors of achievement and aspirations : A multi-cohort study , 2015 .

[44]  Howard D. Bondell,et al.  High Dimensional Linear Regression via the R2-D2 Shrinkage Prior , 2016, 1609.00046.

[45]  Stanislav Kolenikov,et al.  Model-Implied Instrumental Variable—Generalized Method of Moments (MIIV-GMM) Estimators for Latent Variable Models , 2014, Psychometrika.

[46]  J. Eccles,et al.  Expectancy-Value Theory of Achievement Motivation. , 2000, Contemporary educational psychology.

[47]  Muthén Bengt,et al.  Growth Mixture Modeling , 2008, Encyclopedia of Autism Spectrum Disorders.

[48]  J. Spence,et al.  Achievement and achievement motives : psychological and sociological approaches , 1984 .

[49]  Chih-Ling Tsai,et al.  Regression coefficient and autoregressive order shrinkage and selection via the lasso , 2007 .

[50]  G. Casella,et al.  The Bayesian Lasso , 2008 .

[51]  Dieter Zapf,et al.  Advanced Nonlinear Latent Variable Modeling: Distribution Analytic LMS and QML Estimators of Interaction and Quadratic Effects , 2011 .

[52]  A. Kelava,et al.  Estimation of nonlinear latent structural equation models using the extended unconstrained approach , 2009 .

[53]  Chenlei Leng,et al.  Bayesian adaptive Lasso , 2010, Annals of the Institute of Statistical Mathematics.

[54]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[55]  G. Casella,et al.  Penalized regression, standard errors, and Bayesian lassos , 2010 .

[56]  M. Yuan,et al.  Efficient Empirical Bayes Variable Selection and Estimation in Linear Models , 2005 .

[57]  Xiaofang Xu,et al.  Bayesian Variable Selection and Estimation for Group Lasso , 2015, 1512.01013.

[58]  P. Bühlmann,et al.  The group lasso for logistic regression , 2008 .

[59]  K. Schermelleh-Engel,et al.  A Fit Index to Assess Model Fit and Detect Omitted Terms in Nonlinear SEM , 2017 .

[60]  Yue Ma,et al.  Advanced nonlinear structural equation modeling: Theoretical properties and empirical application of the distribution-analytic LMS and QML estimators. , 2011 .

[61]  Yoav Ganzach,et al.  Misleading Interaction and Curvilinear Terms , 1997 .

[62]  James G. Scott,et al.  Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction , 2022 .

[63]  Hongtu Zhu,et al.  Bayesian Lasso for Semiparametric Structural Equation Models , 2012, Biometrics.

[64]  K. Yuan,et al.  5. Three Likelihood-Based Methods for Mean and Covariance Structure Analysis with Nonnormal Missing Data , 2000 .

[65]  Michael O. Martin,et al.  TIMSS 2015 Assessment Frameworks. , 2013 .

[66]  Yasuo Amemiya,et al.  A method of moments technique for fitting interaction effects in structural equation models. , 2003, The British journal of mathematical and statistical psychology.

[67]  Holger Brandt,et al.  The Standardization of Linear and Nonlinear Effects in Direct and Indirect Applications of Structural Equation Mixture Models for Normal and Nonnormal Data , 2015, Front. Psychol..

[68]  Holger Brandt,et al.  A Simulation Study Comparing Recent Approaches for the Estimation of Nonlinear Effects in SEM Under the Condition of Nonnormality , 2014 .

[69]  Thomas Brox,et al.  Maximum Likelihood Estimation , 2019, Time Series Analysis.

[70]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[71]  Helfried Moosbrugger,et al.  Maximum likelihood estimation of latent interaction effects with the LMS method , 2000 .

[72]  Martyn Plummer,et al.  JAGS Version 3.3.0 user manual , 2012 .

[73]  James G. Scott,et al.  The horseshoe estimator for sparse signals , 2010 .

[74]  Ulrich Trautwein,et al.  Probing for the Multiplicative Term in Modern Expectancy-Value Theory: A Latent Interaction Modeling Study. , 2012 .

[75]  Zhen Zhang,et al.  Multilevel structural equation models for assessing moderation within and across levels of analysis. , 2016, Psychological methods.

[76]  H. Watt Development of adolescents' self-perceptions, values, and task perceptions according to gender and domain in 7th- through 11th-grade Australian students. , 2004, Child development.

[77]  Holger Brandt,et al.  A Nonlinear Structural Equation Mixture Modeling Approach for Nonnormally Distributed Latent Predictor Variables , 2014 .

[78]  H. Marsh,et al.  Structural equation models of latent interactions: evaluation of alternative estimation strategies and indicator construction. , 2004, Psychological methods.

[79]  B. Mallick VARIABLE SELECTION FOR REGRESSION MODELS , 2016 .

[80]  Xin-Yuan Song,et al.  Structure detection of semiparametric structural equation models with Bayesian adaptive group lasso , 2015, Statistics in medicine.

[81]  J. S. Rao,et al.  Spike and slab variable selection: Frequentist and Bayesian strategies , 2005, math/0505633.

[82]  Nicholas G. Polson,et al.  The Horseshoe+ Estimator of Ultra-Sparse Signals , 2015, 1502.00560.

[83]  Bengt O. Muthén,et al.  Quasi-Maximum Likelihood Estimation of Structural Equation Models With Multiple Interaction and Quadratic Effects , 2007 .

[84]  J. Griffin,et al.  BAYESIAN HYPER‐LASSOS WITH NON‐CONVEX PENALIZATION , 2011 .

[85]  Yves F. Atchadé,et al.  A computational framework for empirical Bayes inference , 2011, Stat. Comput..

[86]  N. Pillai,et al.  Dirichlet–Laplace Priors for Optimal Shrinkage , 2014, Journal of the American Statistical Association.

[87]  Herbert W. Marsh,et al.  Self-efficacy in classroom management, classroom disturbances, and emotional exhaustion: A moderated mediation analysis of teacher candidates. , 2014 .

[88]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[89]  T. J. Mitchell,et al.  Bayesian Variable Selection in Linear Regression , 1988 .

[90]  J. Friedman,et al.  A Statistical View of Some Chemometrics Regression Tools , 1993 .

[91]  Andrew Gelman,et al.  Handbook of Markov Chain Monte Carlo , 2011 .

[92]  Petros Dellaportas,et al.  On Bayesian model and variable selection using MCMC , 2002, Stat. Comput..