Using Definitive Screening Designs to Identify Active First- and Second-Order Factor Effects

Definitive screening designs (DSDs) were recently introduced by Jones and Nachtsheim (2011b). The use of three-level factors and the desirable aliasing structure of the DSDs make them potentially suitable for identifying main effects and second-order terms in one stage of experimentation. However, as the number of active effects approaches the number of runs, the performance of standard model-selection routines will inevitably degrade. In this paper, we characterize the ability of DSDs to correctly identify first- and second-order model terms as a function of the level of sparsity, the number of factors in the design, the signal-to-noise ratio, the model type (unrestricted or following strong heredity), the model-selection technique, and the number of augmented runs. We find that minimum-run-size DSDs can be used to identify active terms with high probability as long as the number of effects is less than or equal to about half the number of runs and the signal-to-noise ratios for the active effects are above about 2.0. We also find that if minimum-run-size designs are augmented with four or more runs, the number of model terms that can be identified with high probability increases substantially. Among the model-selection methods investigated, we found that both Lasso and the Gauss–Dantzig selector (both based on AICc) can be used to effectively identify active model terms in the presence of unrestricted models. For models following strong heredity, the SHIM method developed by Choi et al. (2010) was the best among methods tested that were designed for the strong-heredity case.