Probabilistic Performance Analysis of Fault Diagnosis Schemes

The dissertation explores the problem of rigorously quantifying the performance of a fault diagnosis scheme in terms of probabilistic performance metrics. Typically, when the performance of a fault diagnosis scheme is of utmost importance, physical redundancy is used to create a highly reliable system that is easy to analyze. However, in this dissertation, we provide a general framework that applies to more complex analytically redundant or model-based fault diagnosis schemes. For each fault diagnosis problem in this framework, our performance metrics can be computed accurately in polynomial-time.First, we cast the fault diagnosis problem as a sequence of hypothesis tests. At each time, the performance of a fault diagnosis scheme is quantified by the probability that the scheme has chosen the correct hypothesis. The resulting performance metrics are joint probabilities. Using Bayes rule, we decompose these performance metrics into two parts: marginal probabilities that quantify the reliability of the system and conditional probabilities that quantify the performance of the fault diagnosis scheme. These conditional probabilities are used to draw connections between the fault diagnosis and the fields of medical diagnostic testing, signal detection, and general statistical decision theory.Second, we examine the problem of computing the performance metrics efficiently and accurately. To solve this problem, we examine each portion of the fault diagnosis problem and specify a set of sufficient assumptions that guarantee efficient computation. In particular, we provide a detailed characterization of the class of finite-state Markov chains that lead to tractable fault parameter models. To demonstrate that these assumptions enable efficient computation, we provide pseudocode algorithms and prove that their running time is indeed polynomial.Third, we consider fault diagnosis problems involving uncertain systems. The inclusion of uncertainty enlarges the class of systems that may be analyzed with our framework. It also addresses the issue of model mismatch between the actual system and the system used to design the fault diagnosis scheme. For various types of uncertainty, we present convex optimization problems that yield the worst-case performance over the uncertainty set.Finally, we discuss applications of the performance metrics and compute the performance for two fault diagnosis problems. The first problem is based on a simplified air-data sensor model, and the second problem is based on a linearized vertical take-off and landing aircraft model.

[1]  Rolf Isermann,et al.  Trends in the Application of Model Based Fault Detection and Diagnosis of Technical Processes , 1996 .

[2]  David J. Hand,et al.  ROC Curves for Continuous Data , 2009 .

[3]  J. B. Pearson,et al.  Performance robustness of discrete-time systems with structured uncertainty , 1990, 29th IEEE Conference on Decision and Control.

[4]  P. Qiu The Statistical Evaluation of Medical Tests for Classification and Prediction , 2005 .

[5]  Peter Seiler,et al.  Performance analysis of LTV fault detection schemes with additive faults , 2011, IEEE Conference on Decision and Control and European Control Conference.

[6]  David J. Hand,et al.  A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems , 2001, Machine Learning.

[7]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[8]  Geir E. Dullerud,et al.  A nonlinear functional approach to LFT model validation , 2002, Syst. Control. Lett..

[9]  Mehrdad Saif,et al.  A new approach to robust fault detection and identification , 1993 .

[10]  Ian Postlethwaite,et al.  Multivariable Feedback Control: Analysis and Design , 1996 .

[11]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[12]  Steven Kay,et al.  Fundamentals Of Statistical Signal Processing , 2001 .

[13]  J. Doyle,et al.  Robust and optimal control , 1995, Proceedings of 35th IEEE Conference on Decision and Control.

[14]  A. Genz,et al.  Computation of Multivariate Normal and t Probabilities , 2009 .

[15]  Ronald L. Graham,et al.  Concrete mathematics - a foundation for computer science , 1991 .

[16]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[17]  Sundeep Rangan,et al.  Model validation for dynamically uncertain systems , 1997 .

[18]  G. Dullerud,et al.  A Course in Robust Control Theory: A Convex Approach , 2005 .

[19]  Y. C. Yeh,et al.  Safety critical avionics for the 777 primary flight controls system , 2001, 20th DASC. 20th Digital Avionics Systems Conference (Cat. No.01CH37219).

[20]  Richard Vernon Beard,et al.  Failure accomodation in linear systems through self-reorganization. , 1971 .

[21]  A. Willsky,et al.  A generalized likelihood ratio approach to the detection and estimation of jumps in linear systems , 1976 .

[22]  H. V. Trees Detection, Estimation, And Modulation Theory , 2001 .

[23]  D. Luenberger Observing the State of a Linear System , 1964, IEEE Transactions on Military Electronics.

[24]  J. Berger Statistical Decision Theory and Bayesian Analysis , 1988 .

[25]  W. H. Chung,et al.  A game theoretic fault detection filter , 1998, IEEE Trans. Autom. Control..

[26]  M. Mariton,et al.  Jump Linear Systems in Automatic Control , 1992 .

[27]  R. E. Kalman,et al.  New Results in Linear Filtering and Prediction Theory , 1961 .

[28]  Edward Chow,et al.  Issues in the development of a general design algorithm for reliable failure detection , 1980, 1980 19th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[29]  Matthew A. Kupinski,et al.  Ideal observers and optimal ROC hypersurfaces in N-class classification , 2004, IEEE Transactions on Medical Imaging.

[30]  Jie Chen,et al.  On computational complexity of invalidating structured uncertainty models 1 1 This research is suppo , 1998 .

[31]  Van Trees,et al.  Detection, Estimation, and Modulation Theory. Part 1 - Detection, Estimation, and Linear Modulation Theory. , 1968 .

[32]  Alan Genz,et al.  Numerical computation of rectangular bivariate and trivariate normal and t probabilities , 2004, Stat. Comput..

[33]  J. Lofberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004, 2004 IEEE International Conference on Robotics and Automation (IEEE Cat. No.04CH37508).

[34]  Mehrdad Saif,et al.  Robust fault isolation observer design , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[35]  R. Patton,et al.  Optimal filtering and robust fault diagnosis of stochastic systems with unknown disturbances , 1996 .

[36]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[37]  K. Poolla,et al.  Spectral power distribution using time-varying operators , 1994, Proceedings of 1994 American Control Conference - ACC '94.

[38]  J. Stoustrup,et al.  Optimal threshold functions for fault detection and isolation , 2003, Proceedings of the 2003 American Control Conference, 2003..

[39]  Peng Shi,et al.  Parity space-based fault detection for Markovian jump systems , 2009, Int. J. Syst. Sci..

[40]  Jason L. Speyer,et al.  Optimal stochastic fault detection filter , 2003, Autom..

[41]  Subir Ghosh,et al.  Reliability and Risk: A Bayesian Perspective , 2008, Technometrics.

[42]  James P. Egan,et al.  Signal detection theory and ROC analysis , 1975 .

[43]  Eric R. Ziegel,et al.  System Reliability Theory: Models, Statistical Methods, and Applications , 2004, Technometrics.

[44]  M. Melamed Detection , 2021, SETI: Astronomy as a Contact Sport.

[45]  Kellen Petersen August Real Analysis , 2009 .

[46]  J. Rosenthal A First Look at Rigorous Probability Theory , 2000 .

[47]  Jie Chen,et al.  Validation of linear fractional uncertain models: solutions via matrix inequalities , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[48]  Gary J. Balas,et al.  Performance analysis of fault detection systems based on analytically redundant linear time-invariant dynamics , 2011, Proceedings of the 2011 American Control Conference.

[49]  Xiao-Hua Zhou,et al.  Statistical Methods in Diagnostic Medicine , 2002 .

[50]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[51]  Chi-Tsong Chen,et al.  Linear System Theory and Design , 1995 .

[52]  Xin He,et al.  The Meaning and Use of the Volume Under a Three-Class ROC Surface (VUS) , 2008, IEEE Transactions on Medical Imaging.

[53]  Ian Moir,et al.  Civil Avionics Systems , 2002 .

[54]  Aarnout Brombacher,et al.  Probability... , 2009, Qual. Reliab. Eng. Int..

[55]  Chang‐Jin Kim,et al.  State-Space Models with Regime-Switching: Classical and Gibbs Sampling Approaches with Applications , 1999 .

[56]  A. Ray,et al.  An introduction to sensor signal validation in redundant measurement systems , 1991, IEEE Control Systems.

[57]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[58]  G. Yin,et al.  Hybrid Switching Diffusions: Properties and Applications , 2009 .

[59]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[60]  Stephen E. Fienberg,et al.  Testing Statistical Hypotheses , 2005 .

[61]  Steven X. Ding,et al.  Optimal fault detection for linear discrete time-varying systems , 2010, Autom..

[62]  K. Narendra,et al.  Identification and Optimization of Aircraft Dynamics , 1973 .

[63]  Audra E. Kosh,et al.  Linear Algebra and its Applications , 1992 .

[64]  José Hernández-Orallo,et al.  Volume under the ROC Surface for Multi-class Problems , 2003, ECML.

[65]  Jason L. Speyer,et al.  Robust fault detection filter design , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[66]  Kameshwar Poolla,et al.  Robust performance against slowly-varying structured perturbations , 1993, Proceedings of 32nd IEEE Conference on Decision and Control.

[67]  R. P. Marques,et al.  Discrete-Time Markov Jump Linear Systems , 2004, IEEE Transactions on Automatic Control.

[68]  Jie Chen,et al.  Design of unknown input observers and robust fault detection filters , 1996 .

[69]  R. K. Mehra,et al.  Correspondence item: An innovations approach to fault detection and diagnosis in dynamic systems , 1971 .

[70]  Lotfi A. Zadeh,et al.  Optimality and non-scalar-valued performance criteria , 1963 .

[71]  Johan Efberg,et al.  YALMIP : A toolbox for modeling and optimization in MATLAB , 2004 .

[72]  H. Vincent Poor,et al.  An Introduction to Signal Detection and Estimation , 1994, Springer Texts in Electrical Engineering.

[73]  Jie Chen,et al.  Robust Model-Based Fault Diagnosis for Dynamic Systems , 1998, The International Series on Asian Studies in Computer and Information Science.

[74]  Rolf Isermann,et al.  Process fault detection based on modeling and estimation methods - A survey , 1984, Autom..

[75]  Jonathan E. Fieldsend,et al.  Multi-class ROC analysis from a multi-objective optimisation perspective , 2006, Pattern Recognit. Lett..

[76]  Dominique Sauter,et al.  Robust fault detection in uncertain dynamic systems , 2000, Autom..

[77]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[78]  M. Mariton Detection delays, false alarm rates and the reconfiguration of control systems , 1989 .

[79]  Bernard C. Levy,et al.  Principles of Signal Detection and Parameter Estimation , 2008 .

[80]  Peter W. Glynn,et al.  Stochastic Simulation: Algorithms and Analysis , 2007 .

[81]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[82]  Pramod Khargonekar,et al.  A Time-Domain Approach to Model Validation , 1992, 1992 American Control Conference.

[83]  M. Degroot Optimal Statistical Decisions , 1970 .

[84]  James Lam,et al.  Worst-Case Fault Detection Observer Design: Optimization Approach , 2007 .

[85]  Nozer D. Singpurwalla,et al.  Reliability and risk , 2006 .

[86]  Steven X. Ding,et al.  Model-based Fault Diagnosis Techniques: Design Schemes, Algorithms, and Tools , 2008 .

[87]  David Williams,et al.  Probability with Martingales , 1991, Cambridge mathematical textbooks.

[88]  Yehoshua Bar-Hillel,et al.  The Intrinsic Computational Difficulty of Functions , 1969 .

[89]  J. Edmonds Paths, Trees, and Flowers , 1965, Canadian Journal of Mathematics.

[90]  R. W. Wilde,et al.  Observers for linear systems with unknown inputs , 1988 .

[91]  A. Shiryaev On Optimum Methods in Quickest Detection Problems , 1963 .

[92]  P. Gács,et al.  Algorithms , 1992 .

[93]  Heng Wang,et al.  Fault Detection Observer Design in Low Frequency Domain , 2007, 2007 IEEE International Conference on Control Applications.

[94]  A. Willsky,et al.  Analytical redundancy and the design of robust failure detection systems , 1984 .

[95]  Y. C. Yeh,et al.  Triple-triple redundant 777 primary flight computer , 1996, 1996 IEEE Aerospace Applications Conference. Proceedings.

[96]  James Lam,et al.  Robust Fault Detection of Markovian Jump Systems , 2004 .

[97]  Takashi Yoneyama,et al.  A Bayesian solution to the multiple composite hypothesis testing for fault diagnosis in dynamic systems , 2011, Autom..

[98]  Qiang Luo,et al.  Robust isolable models for failure diagnosis , 1989 .

[99]  M. M. Akhter,et al.  Effect of model uncertainty on failure detection: the threshold selector , 1988 .

[100]  S. Kay Fundamentals of statistical signal processing: estimation theory , 1993 .

[101]  Michel Verhaegen,et al.  Robust fault detection observer design for linear uncertain systems , 2011, Int. J. Control.

[102]  Michèle Basseville,et al.  Detection of abrupt changes: theory and application , 1993 .

[103]  R. P. G. Collinson,et al.  Introduction to Avionics Systems , 2003 .

[104]  W. Cody,et al.  Rational Chebyshev approximations for the error function , 1969 .

[105]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .