Explaining Probabilistic Fault Diagnosis and Classification Using Case-Based Reasoning

This paper describes a generic framework for explaining the prediction of a probabilistic classifier using preceding cases. Within the framework, we derive similarity metrics that relate the similarity between two cases to a probability model and propose a novel case-based approach to justifying a classification using the local accuracy of the most similar cases as a confidence measure. As a basis for deriving similarity metrics, we define similarity in terms of the principle of interchangeability that two cases are considered similar or identical if two probability distributions, derived from excluding either one or the other case in the case base, are identical. Thereafter, we evaluate the proposed approach for explaining the probabilistic classification of faults by logistic regression. We show that with the proposed approach, it is possible to find cases for which the used classifier accuracy is very low and uncertain, even though the predicted class has high probability.

[1]  Shunsuke Ihara,et al.  Information theory - for continuous systems , 1993 .

[2]  Rolf Isermann,et al.  Supervision, fault-detection and fault-diagnosis methods — An introduction , 1997 .

[3]  Agnar Aamodt,et al.  Explanation-Driven Case-Based Reasoning , 1993, EWCBR.

[4]  Charles F. Hockett,et al.  A mathematical theory of communication , 1948, MOCO.

[5]  Edward H. Shortliffe,et al.  Adapting a Consultation System to Critique User Plans , 1983, Int. J. Man Mach. Stud..

[6]  L. Richard Ye,et al.  The Impact of Explanation Facilities in User Acceptance of Expert System Advice , 1995, MIS Q..

[7]  Derek G. Bridge,et al.  KLEOR: A Knowledge Lite Approach to Explanation Oriented Retrieval , 2006, Comput. Artif. Intell..

[8]  Tomas Olsson,et al.  Case-Based Reasoning for Explaining Probabilistic Machine Learning , 2014 .

[9]  Hans-Dieter Burkhard.  Similarity and Distance in Case Based Reasoning , 2001, Fundam. Informaticae.

[10]  Padraig Cunningham,et al.  A Case-Based Explanation System for Black-Box Systems , 2005, Artificial Intelligence Review.

[11]  John David N. Dionisio,et al.  Case-based explanation of non-case-based learning methods , 1999, AMIA.

[12]  Peter Funk,et al.  Fault diagnosis in industry using sensor readings and case-based reasoning , 2004, J. Intell. Fuzzy Syst..

[13]  David McSherry,et al.  Introduction to the Special Issue on Explanation in Case-Based Reasoning , 2005, Artificial Intelligence Review.

[14]  Tomas Olsson,et al.  Case-based reasoning combined with statistics for diagnostics and prognosis , 2012 .

[15]  Alexey Tsymbal,et al.  A Review of Explanation and Explanation in Case-Based Reasoning , 2003 .

[16]  Roger C. Schank,et al.  Creativity and Learning in a Case-Based Explainer , 1989, Artif. Intell..

[17]  Padraig Cunningham,et al.  Explaining Predictions from a Neural Network Ensemble One at a Time , 2002, PKDD.

[18]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[19]  Padraig Cunningham,et al.  An Evaluation of the Usefulness of Case-Based Explanation , 2003, ICCBR.

[20]  Izak Benbasat,et al.  Explanations From Intelligent Systems: Theoretical Foundations and Implications for Practice , 1999, MIS Q..

[21]  Sung-Hyuk Cha.  Comprehensive Survey on Distance/Similarity Measures between Probability Density Functions , 2007 .

[22]  Padraig Cunningham,et al.  Explanation Oriented Retrieval , 2004, ECCBR.

[23]  Padraig Cunningham,et al.  The Best Way to Instil Confidence Is by Being Right , 2005, ICCBR.

[24]  David McSherry,et al.  A Lazy Learning Approach to Explaining Case-Based Reasoning Solutions , 2012, ICCBR.

[25]  Keith Darlington,et al.  Aspects of Intelligent Systems Explanation , 2013 .

[26]  Michael M. Richter,et al.  On the Notion of Similarity in Case Based Reasoning and Fuzzy Theory , 2001, Soft Computing in Case Based Reasoning.

[27]  Michael Green,et al.  Exploring new possibilities for case-based explanation of artificial neural network ensembles , 2009, Neural Networks.

[28]  Mattias Ohlsson,et al.  Explaining artificial neural network ensembles: A case study with electrocardiograms from chest pain patients , 2008, ICML 2008.

[29]  Stoyan V. Stoyanov,et al.  A Probability Metrics Approach to Financial Risk Measures: Rachev/A Probability Metrics Approach to Financial Risk Measures , 2011 .

[30]  A. K. Wadhwani,et al.  Machine Fault Signature Analysis , 2008 .

[31]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[32]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[33]  Jihong Lee,et al.  EKF-based Localization of a Mobile Robot using Pattern Matching , 2013 .

[34]  Michael I. Jordan,et al.  On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes , 2001, NIPS.

[35]  William B. Thompson,et al.  Reconstructive Expert System Explanation , 1992, Artif. Intell..

[36]  Jianhua Lin,et al.  Divergence measures based on the Shannon entropy , 1991, IEEE Trans. Inf. Theory.

[37]  Erik Olsson.  Fault Diagnosis of Industrial Machines using Sensor Signals and Case-Based Reasoning , 2009 .

[38]  Dónal Doyle,et al.  An Evaluation of the Usefulness oof Case-Based Reasoning Explanation , 2003 .

[39]  David McSherry,et al.  Explaining the Pros and Cons of Conclusions in CBR , 2004, ECCBR.

[40]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .