A comparison of design and model selection methods for supersaturated experiments

Various design and model selection methods are available for supersaturated designs having more factors than runs but little research is available on their comparison and evaluation. Simulated experiments are used to evaluate the use of E(s^2)-optimal and Bayesian D-optimal designs and to compare three analysis strategies representing regression, shrinkage and a novel model-averaging procedure. Suggestions are made for choosing the values of the tuning constants for each approach. Findings include that (i) the preferred analysis is via shrinkage; (ii) designs with similar numbers of runs and factors can be effective for a considerable number of active effects of only moderate size; and (iii) unbalanced designs can perform well. Some comments are made on the performance of the design and analysis methods when effect sparsity does not hold.

[1]  S. Lewis,et al.  Detection of interactions in experiments on large numbers of factors , 2001 .

[2]  Dennis K. J. Lin,et al.  Bayesian D-optimal supersaturated designs , 2008 .

[3]  F. E. Satterthwaite.  Random Balance Experimentation , 1959 .

[4]  R. Daniel Meyer,et al.  An Analysis for Unreplicated Fractional Factorials , 1986 .

[5]  Dennis K. J. Lin,et al.  Data analysis in supersaturated designs , 2002 .

[6]  Steven G. Gilmour,et al.  Factor Screening via Supersaturated Designs , 2006 .

[7]  Theodore T. Allen,et al.  Supersaturated Designs That Maximize the Probability of Identifying Active Factors , 2003, Technometrics.

[8]  Anthony O'Hagan,et al.  Kendall's Advanced Theory of Statistics: Vol. 2B, Bayesian Inference. , 1996 .

[9]  H. Chipman,et al.  Some risks in the construction and analysis of supersaturated designs , 1999 .

[10]  W. DuMouchel,et al.  A simple Bayesian modification of D-optimal designs to reduce dependence on an assumed model , 1994 .

[11]  Frederick Kin Hing Phoa,et al.  Analysis of Supersaturated Designs via Dantzig Selector , 2009 .

[12]  R. K. Meyer,et al.  The Coordinate-Exchange Algorithm for Constructing Exact Optimal Experimental Designs , 1995 .

[13]  Dursun A. Bulutoglu,et al.  E(s2)-Optimal supersaturated designs with good minimax properties , 2007 .

[14]  Alan J. Miller.  Subset Selection in Regression , 1992 .

[15]  C. D. Kemp,et al.  Kendall's Advanced Theory of Statistics, Vol. 1: Distribution Theory. , 1995 .

[16]  Changbao Wu,et al.  Construction of supersaturated designs through partially aliased interactions , 1993 .

[17]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[18]  David R. Anderson,et al.  Model Selection and Multimodel Inference , 2003 .

[19]  William Li,et al.  Columnwise-pairwise algorithms with applications to the construction of supersaturated designs , 1997 .

[20]  Dennis K. J. Lin,et al.  A new class of supersaturated designs , 1993 .

[21]  K. H. Booth,et al.  Some Systematic Supersaturated Designs , 1962 .

[22]  D. Madigan,et al.  Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window , 1994 .

[23]  Runze Li,et al.  Analysis Methods for Supersaturated Design: Some Comparisons , 2003, Journal of Data Science.

[24]  Dennis K. J. Lin,et al.  A Two-Stage Bayesian Model Selection Strategy for Supersaturated Designs , 2002, Technometrics.

[25]  D. Ankerst,et al.  Kendall's Advanced Theory of Statistics, Vol. 2B: Bayesian Inference , 2005 .

[26]  Dennis K. J. Lin,et al.  FORWARD SELECTION ERROR CONTROL IN THE ANALYSIS OF SUPERSATURATED DESIGNS , 1998 .

[27]  Nam-Ky Nguyen,et al.  New E(S2)-Optimal Supersaturated Designs Constructed From Incomplete Block Designs , 2008, Technometrics.

[28]  A. O'Hagan,et al.  Kendall's Advanced Theory of Statistics, Vol. 2b: Bayesian Inference. , 1996 .

[29]  Douglas C. Montgomery,et al.  The Use of Supersaturated Experiments in Turbine Engine Development , 2007 .

[30]  Nam-Ky Nguyen,et al.  New E ( s 2 )-Optimal Supersaturated Designs Constructed From Incomplete Block Designs , 2008 .

[31]  K. Vijayan,et al.  Some Risks in the Construction and Analysis of Supersaturated Designs , 1999, Technometrics.

[32]  Nam-Ky Nguyen.  An algorithmic approach to constructing supersaturated designs , 1996 .

[33]  Dennis K. J. Lin.  Generating Systematic Supersaturated Designs , 1995 .