Performance Bounds for Neural Network Estimators: Applications in Fault Detection

We exploit recent results in quantifying the robustness of neural networks to input variations to construct and tune a model-based anomaly detector, where the data-driven estimator model is provided by an autoregressive neural network. In tuning, we specifically provide upper bounds on the rate of false alarms expected under normal operation. To accomplish this, we provide a theory extension to allow for the propagation of multiple confidence ellipsoids through a neural network. The ellipsoid that bounds the output of the neural network under the input variation informs the sensitivity - and thus the threshold tuning - of the detector. We demonstrate this approach on a linear and nonlinear dynamical system.

[1]  M.H. Hassoun,et al.  Fundamentals of Artificial Neural Networks , 1996, Proceedings of the IEEE.

[2]  Paul M. Frank,et al.  New developments using AI in fault diagnosis , 1996 .

[3]  Manfred Morari,et al.  Probabilistic Verification and Reachability Analysis of Neural Networks via Semidefinite Programming , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[4]  Navid Hashemi,et al.  Certifying Incremental Quadratic Constraints for Neural Networks. , 2020 .

[5]  Aditi Raghunathan,et al.  Semidefinite relaxations for certifying robustness to adversarial examples , 2018, NeurIPS.

[6]  Alessio Lomuscio,et al.  An approach to reachability analysis for feed-forward ReLU neural networks , 2017, ArXiv.

[7]  Khashayar Khorasani,et al.  Fault Detection in Reaction Wheel of a Satellite Using Observer-Based Dynamic Neural Networks , 2005, ISNN.

[8]  H.A. Toliyat,et al.  Artificial-neural-network-based sensorless nonlinear control of induction motors , 2005, IEEE Transactions on Energy Conversion.

[9]  Carlos Murguia,et al.  On model‐based detectors for linear time‐invariant stochastic systems under sensor attacks , 2019, IET Control Theory & Applications.

[10]  Wei Liu.  An extended Kalman filter and neural network cascade fault diagnosis strategy for the glutamic acid fermentation process , 1999, Artif. Intell. Eng..

[11]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[12]  John R. Hershey,et al.  Uncertainty propagation through deep neural networks , 2015, INTERSPEECH.

[13]  Ashish Tiwari,et al.  Output Range Analysis for Deep Neural Networks , 2017, ArXiv.

[14]  W. Marsden.  I and J , 2012 .

[15]  Jeremy Kepner,et al.  Uncertainty Propagation in Deep Neural Networks Using Extended Kalman Filtering , 2018, 2018 IEEE MIT Undergraduate Research Technology Conference (URTC).

[16]  Marios M. Polycarpou,et al.  Neural-network-based robust fault diagnosis in robotic systems , 1997, IEEE Trans. Neural Networks.

[17]  Manfred Morari,et al.  Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programming , 2019, ArXiv.

[18]  Arman Sargolzaei,et al.  Neural adaptive observer-based sensor and actuator fault detection in nonlinear systems: Application in UAV. , 2017, ISA transactions.

[19]  Mykel J. Kochenderfer,et al.  Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks , 2017, CAV.

[20]  Min Wu,et al.  Safety Verification of Deep Neural Networks , 2016, CAV.

[21]  Navid Hashemi,et al.  Co-design for Performance and Security: Geometric Tools , 2020 .

[22]  J. Zico Kolter,et al.  Provable defenses against adversarial examples via the convex outer adversarial polytope , 2017, ICML.