An Adjusted Bonferroni Method for Elimination of Parameters in Specification Addition Searches

An adjusted Bonferroni method for controlling familywise error rate when deleting parameters in specification addition searches was presented and evaluated in a 2-part study. First, data were generated based on a factor model and then analyzed using backward selection, with and without applying the adjusted Bonferroni method. Three factors were manipulated: sample size, magnitude of weights, and number of parameters in a search. Under most of the 30 explored conditions, the empirical familywise error rates were relatively close to the nominal alpha level of. 05 when the adjusted Bonferroni method was applied. Error rates that exceeded the. 05 level appeared to be a function of the multivariate Wald test and not of the adjusted Bonferroni method. Second, data were generated based on a path model and analyzed using 2-stage and backward selection methods when the initial model was misspecified. Controlling stringently for Type I errors in the initial addition stage of a 2-stage search created selection errors when N was small (200). The adjusted Bonferroni method controlled adequately for Type I errors in the deletion stage of the 2-stage search and with backward selection.

[1]  Samuel B. Green,et al.  Exploratory analyses to improve model fit: Errors due to misspecification and a strategy to reduce their occurrence , 1999 .

[2]  S. Green,et al.  Control of Type I Errors with Multiple Tests of Constraints in Structural Equation Modeling. , 1997, Multivariate behavioral research.

[3]  A. Satorra.  Alternative test criteria in covariance structure analysis: A unified approach , 1989 .

[4]  P. Bentler,et al.  Invariant Standardized Estimated Parameter Change for Model Modification in Covariance Structure Analysis. , 1993, Multivariate behavioral research.

[5]  Peter M. Bentler,et al.  EQS : structural equations program manual , 1989 .

[6]  D. Sörbom.  Model modification , 1989 .

[7]  L. Harlow,et al.  Effects of estimation methods, number of indicators per factor, and improper solutions on structural equation modeling fit indices , 1995 .

[8]  P. Bentler,et al.  Model Modification in Covariance Structure Modeling: A Comparison among Likelihood Ratio, Lagrange Multiplier, and Wald Tests. , 1990, Multivariate behavioral research.

[9]  D Kaplan,et al.  The Impact of Specification Error on the Estimation, Testing, and Improvement of Structural Equation Models. , 1988, Multivariate behavioral research.

[10]  Zvi Drezner,et al.  Model specification searches in structural equation modeling using tabu search , 1998 .

[11]  S. Green,et al.  A Monte Carlo Investigation of Methods for Controlling Type I Errors with Specification Searches in Structural Equation Modeling. , 1998, Multivariate behavioral research.

[12]  R. MacCallum,et al.  Model modifications in covariance structure analysis: the problem of capitalization on chance. , 1992, Psychological bulletin.

[13]  D. Kaplan.  Evaluating and Modifying Covariance Structure Models: A Review and Recommendation. , 1990, Multivariate behavioral research.

[14]  Karl G. Jöreskog,et al.  LISREL 7: A guide to the program and applications , 1988 .

[15]  C. Chou,et al.  Some New Covariance Structure Model Improvement Statistics , 1992 .

[16]  Robert C. MacCallum,et al.  SPECIFICATION SEARCHES IN COVARIANCE STRUCTURE MODELING , 1986 .

[17]  Gregory R. Hancock.  A sequential Scheffé‐type respecification procedure for controlling type I error in exploratory structural equation model modification , 1999 .

[18]  Tenko Raykov,et al.  Issues in applied structural equation modeling research , 1995 .

[19]  N. Mantel.  Why Stepdown Procedures in Variable Selection , 1970 .

[20]  James C. Anderson,et al.  Improper solutions in the analysis of covariance structures: Their interpretability and a comparison of alternate respecifications , 1987 .

[21]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .