Comprehensive comparisons of major sequential design procedures for sensitivity testing

Abstract The main goal of a good sensitivity testing procedure is to quickly determine at which stimulus level a specified fraction of test items fail. Based on a previous study, two procedures, D-optimal and 3pod, appear to be most promising. In this paper we expand on this work, first by considering other variants of the two procedures. A related procedure to D-optimal, called Sen-Test, is examined and shown to be superior to D-optimal in a simulation study. A revised version of 3pod, dubbed 3pod2.0, is proposed. A comprehensive simulation study to compare Sen-Test, 3pod and 3pod2.0 is conducted. Based on this study, we draw some concluding remarks on their relative performance and make recommendations.

[1]  C. F. Wu,et al.  Efficient Sequential Designs with Binary Data , 1985 .

[2]  Barry T. Neyer,et al.  A D-Optimality-Based Sensitivity Test , 1994 .

[3]  Paul A. Roediger.  Gonogo: An R Implementation of Test Methods to Perform, Analyze and Simulate Sensitivity Experiments , 2020 .

[4]  J. O. Irwin,et al.  Statistical Method in Biological Assay , 1953, Nature.

[5]  V. R. Joseph,et al.  ADAPTIVE DESIGNS FOR STOCHASTIC ROOT-FINDING , 2007 .

[6]  V. Roshan Joseph,et al.  Efficient Robbins-Monro procedure for binary data , 2004 .

[7]  Commentary: Three-phase optimal design for sensitivity experiments , 2014 .

[8]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[9]  D. Cox,et al.  Analysis of Binary Data (2nd ed.). , 1990 .

[10]  David R. Cox.  The analysis of binary data , 1970 .

[11]  Douglas M. Ray,et al.  Adaptive Testing of DoD Systems with Binary Response , 2018 .

[12]  Three-phase optimal design of sensitivity experiments , 2014 .

[13]  Fred Spiring,et al.  Introduction to Statistical Quality Control , 2007, Technometrics.

[14]  J. Sacks.  Asymptotic Distribution of Stochastic Approximation Procedures , 1958 .

[15]  Alexander M. Mood,et al.  A Method for Obtaining and Analyzing Sensitivity Data , 1948 .

[16]  Hovav A. Dror,et al.  Sequential Experimental Designs for Generalized Linear Models , 2008 .

[17]  D. W. Cooper.  Adaptive testing , 1976, ICSE '76.

[18]  H J Langlie,et al.  A Reliability Test Method for "One-Shot" Items , 1963 .

[19]  Laura J. Freeman,et al.  A Comparison of Ballistic Resistance Testing Techniques in the Department of Defense , 2014, IEEE Access.