Is Average Run Length to False Alarm Always an Informative Criterion?

Abstract Apart from Bayesian approaches, the average run length (ARL) to false alarm has always been seen as the natural performance criterion for quantifying the propensity of a detection scheme to make false alarms, and no researchers seem to have questioned this on grounds that it does not always apply. In this article, we show that in the change-point problem with mixture prechange models, detection schemes with finite detection delays can have infinite ARLs to false alarm. We also discuss the implication of our results on the change-point problem with either exchangeable prechange models or hidden Markov models. Alternative minimax formulations with different false alarm criteria are proposed.

[1]  George V. Moustakides,et al.  Optimality of the CUSUM procedure in continuous time , 2003 .

[2]  David Blackwell,et al.  On an Equation of Wald , 1946 .

[3]  York Marcel Dekker.  The State of Statistical Process Control as We Proceed into the 21st Century , 2000 .

[4]  Boris Darkhovsky,et al.  Asymptotically optimal methods of change-point detection for composite hypotheses , 2005 .

[5]  B. E. Brodsky,et al.  Non-Parametric Statistical Diagnosis: Problems and Methods , 2000 .

[6]  G. Barnard.  Control Charts and Stochastic Processes , 1959 .

[7]  Martin Kulldorff,et al.  Prospective time periodic geographical disease surveillance using a scan statistic , 2001 .

[8]  A. R. Crathorne,et al.  Economic Control of Quality of Manufactured Product. , 1933 .

[9]  Benjamin Yakir.  A lower bound on the ARL to detection of a change with a probability constraint on false alarm , 1996 .

[10]  M. Pollak.  Average Run Lengths of an Optimal Method of Detecting a Change in Distribution. , 1987 .

[11]  T. Lai.  SEQUENTIAL ANALYSIS: SOME CLASSICAL PROBLEMS AND NEW CHALLENGES , 2001 .

[12]  Inchi Hu,et al.  Invited Discussion of "Sequential Analysis: Some Classical Problems and New Challenges , 2001 .

[13]  Alexander G. Tartakovsky,et al.  Asymptotic Optimality of Change-Point Detection Schemes in General Continuous-Time Models , 2006 .

[14]  D. Siegmund,et al.  Sequential Detection of a Change in a Normal Mean when the Initial Value is Unknown , 1991 .

[15]  Peter Willett,et al.  Multisensor Track Termination for Targets with Fluctuating SNR , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[16]  Y. Ritov.  Decision Theoretic Optimality of the Cusum Procedure , 1990 .

[17]  William H. Woodall,et al.  The Distribution of the Run Length of One-Sided CUSUM Procedures for Continuous Random Variables , 1983 .

[18]  Tze Leung Lai,et al.  Importance Sampling for Generalized Likelihood Ratio Procedures in Sequential Analysis , 2005 .

[19]  Moshe Pollak,et al.  Asymptotic Exponentiality of First Exit Times for Recurrent Markov Processes and Applications to Changepoint Detection , 2008 .

[20]  G. Moustakides.  Optimal stopping times for detecting changes in distributions , 1986 .

[21]  Benjamin Yakir,et al.  Minimax optimality of the Shiryayev–Roberts change-point detection rule , 2008 .

[22]  A. Tartakovsky.  Asymptotic Performance of a Multichart CUSUM Test Under False Alarm Probability Constraint , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[23]  Zeev Barzily,et al.  Bayes procedures for detecting a shift in the probability of success in a series of Bernoulli trials , 1981 .

[24]  Moshe Pollak,et al.  Detecting a change in regression: first-order optimality , 1999 .

[25]  Galit Shmueli,et al.  Statistical issues and challenges associated with rapid detection of bio‐terrorist attacks , 2005, Statistics in medicine.

[26]  William H. Woodall,et al.  The Use of Control Charts in Health-Care and Public-Health Surveillance , 2006 .

[27]  C. Fuh.  Asymptotic operating characteristics of an optimal change point detection in hidden Markov models , 2004, math/0503682.

[28]  D. Iglehart.  Extreme Values in the GI/G/1 Queue , 1972 .

[29]  B. Brodsky,et al.  Nonparametric Methods in Change Point Problems , 1993 .

[30]  Cheng-Der Fuh,et al.  SPRT and CUSUM in hidden Markov models , 2003 .

[31]  D. Siegmund.  Sequential Analysis: Tests and Confidence Intervals , 1985 .

[32]  Y. Mei.  Sequential change-point detection when unknown parameters are present in the pre-change distribution , 2006, math/0605322.

[33]  T. Lai.  Sequential changepoint detection in quality control and dynamical systems , 1995 .

[34]  Rudolf B. Blazek,et al.  Detection of intrusions in information systems by sequential change-point methods , 2005 .

[35]  Tze Leung Lai,et al.  Information Bounds and Quick Detection of Parameter Changes in Stochastic Systems , 1998, IEEE Trans. Inf. Theory.

[36]  Michèle Basseville,et al.  Detection of abrupt changes: theory and application , 1993 .

[37]  Charles W. Champ,et al.  Effects of Parameter Estimation on Control Chart Properties: A Literature Review , 2006 .

[38]  Tze Leung Lai,et al.  Efficient recursive algorithms for detection of abrupt changes in signals and control systems , 1999, IEEE Trans. Autom. Control..

[39]  S. W. Roberts.  A Comparison of Some Control Chart Procedures , 1966 .

[40]  G. Moustakides.  SEQUENTIAL CHANGE DETECTION REVISITED , 2008, 0804.0741.

[41]  Michael Woodroofe,et al.  Nonlinear Renewal Theory , 2006 .

[42]  K. Waldmann,et al.  Bounds for the Distribution of the Run Length of One-Sided and Two-Sided CUSUM Quality Control Schemes , 1986 .

[43]  B. E. Brodsky,et al.  Non-Parametric Statistical Diagnosis , 2000 .

[44]  Surveillance of a Simple Linear Regression , 2003 .

[45]  L. J. Savage,et al.  Symmetric measures on Cartesian products , 1955 .

[46]  Benjamin Yakir,et al.  Optimal detection of a change in distribution when the observations form a Markov chain with a finite state space , 1994 .

[47]  Moshe Pollak,et al.  Asymptotic Exponentiality of the Distribution of First Exit Times for a Class of Markov Processes with Applications to Quickest Change Detection , 2006 .

[48]  E. S. Page.  CONTINUOUS INSPECTION SCHEMES , 1954 .

[49]  V. Veeravalli,et al.  General Asymptotic Bayesian Theory of Quickest Change Detection , 2005 .

[50]  Michèle Basseville,et al.  Detection of Abrupt Changes: Theory and Applications. , 1995 .

[51]  Yajun Mei.  SUBOPTIMAL PROPERTIES OF PAGE'S CUSUM AND SHIRYAYEV-ROBERTS PROCEDURES IN CHANGE-POINT PROBLEMS WITH DEPENDENT OBSERVATIONS , 2006 .

[52]  A. Shiryaev.  On Optimum Methods in Quickest Detection Problems , 1963 .

[53]  R. Berk.  STOPPING TIMES OF SPRTS BASED ON EXCHANGEABLE MODELS , 1970 .

[54]  Stephen A. Freitas,et al.  Modern Industrial Statistics: Design and Control of Quality and Reliability , 1999, Technometrics.

[55]  D. Siegmund,et al.  APPROXIMATIONS TO THE EXPECTED SAMPLE SIZE OF CERTAIN SEQUENTIAL TESTS , 1975 .

[56]  Moshe Pollak,et al.  ON OPTIMALITY PROPERTIES OF THE SHIRYAEV-ROBERTS PROCEDURE , 2007, 0710.5935.

[57]  N. Urban,et al.  A parametric empirical Bayes method for cancer screening using longitudinal observations of a biomarker. , 2003, Biostatistics.

[58]  T. Lai,et al.  Efficient importance sampling for Monte Carlo evaluation of exceedance probabilities , 2007, math/0703910.

[59]  D. Siegmund,et al.  Using the Generalized Likelihood Ratio Statistic for Sequential Detection of a Change-Point , 1995 .

[60]  Rakesh K. Bansal,et al.  An algorithm for detecting a change in a stochastic process , 1986, IEEE Trans. Inf. Theory.

[61]  M. Woodroofe.  Nonlinear Renewal Theory in Sequential Analysis , 1987 .

[62]  M. Pollak.  Optimal Detection of a Change in Distribution , 1985 .

[63]  G. Lorden.  PROCEDURES FOR REACTING TO A CHANGE IN DISTRIBUTION , 1971 .

[64]  M. Beibel,et al.  Sequential change-point detection in continuous time when the post-change drift is unknown , 1997 .

[65]  J. Kiefer,et al.  Asymptotically Optimum Sequential Inference and Design , 1963 .

[66]  Shelemyahu Zacks.  Exact Determination of the Run Length Distribution of a One-Sided CUSUM Procedure Applied on an Ordinary Poisson Process , 2004 .

[67]  B. Brodsky,et al.  Minimax Methods for Multihypothesis Sequential Testing and Change-Point Detection Problems , 2008 .

[68]  D. Siegmund,et al.  A diffusion process and its applications to detecting a change in the drift of Brownian motion , 1984 .

[69]  Alʹbert Nikolaevich Shiri︠a︡ev,et al.  Optimal stopping rules , 1977 .

[70]  Shelemyahu Zacks.  The probability distribution and the expected value of a stopping variable associated with one-sided cusum procedures for non-negative integer valued random variables , 1981 .

[71]  Marianne Frisén,et al.  Optimal Surveillance Based on Exponentially Weighted Moving Averages , 2006 .

[72]  H. Poor.  Quickest detection with exponential penalty for delay , 1998 .