Robust learning in safety related domains: machine learning methods for solving safety related application problems

Today, machine learning methods are successfully deployed in a wide range of applications. A multitude of different learning algorithms has been developed in order to solve classification and regression problems. These common machine learning approaches are regarded with suspicion by domain experts in safety-related application fields because it is often infeasible to sufficiently interpret and validate the learned solutions. Especially for safety-related applications, it is imperative to guarantee that the learned solution is correct and fulfills all given requirements. The basic idea of the approaches proposed within this thesis is to solve high-dimensional application problems by an ensemble of simple submodels, each of which is allowed to only use two or three dimensions of the complete input space. The restriction of the dimensionality of the submodels allows the visualization of the learned models. Thus a visual interpretation and validation according to the existing domain knowledge becomes feasible. Due to the visualization, an unintended and possibly undesired extraand interpolation behavior can be discovered and avoided by changing the model parameters or selecting other submodels. Since the learned submodels are interpretable the correctness of the learned solution can therefore be guaranteed. The ensemble of the submodels compensates for the limited dimensionality of the individual submodels. The proposed ensemble methods are successfully applied on common benchmark problems as well as on real-world application problems with very high requirements on the functional safety of the learned solution.

[1]  Detlef Nauck,et al.  Foundations Of Neuro-Fuzzy Systems , 1997 .

[2]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[3]  Development Guidelines for Vehicle Based Software , 2022 .

[4]  Simon Kasif,et al.  A System for Induction of Oblique Decision Trees , 1994, J. Artif. Intell. Res..

[5]  Masoud Nikravesh,et al.  Feature Extraction - Foundations and Applications , 2006, Feature Extraction.

[6]  Rudolf Kruse,et al.  Obtaining interpretable fuzzy classification rules from medical data , 1999, Artif. Intell. Medicine.

[7]  Tim Kelly,et al.  Using fuzzy self-organising maps for safety critical systems , 2007, Reliab. Eng. Syst. Saf..

[8]  David J. Hand,et al.  A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems , 2001, Machine Learning.

[9]  Kwan-Liu Ma,et al.  Opening the black box - data driven visualization of neural networks , 2005, VIS 05. IEEE Visualization, 2005..

[10]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[11]  W. M. Thorburn,et al.  THE MYTH OF OCCAM'S RAZOR , 1918 .

[12]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[13]  Werner Hauptmann,et al.  An EM-Based Piecewise Linear Regression Algorithm , 2008, HAIS.

[14]  J. Morgan,et al.  Problems in the Analysis of Survey Data, and a Proposal , 1963 .

[15]  Peter A. Flach,et al.  Improving Accuracy and Cost of Two-class and Multi-class Probabilistic Classifiers Using ROC Curves , 2003, ICML.

[16]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[17]  Ian T. Nabney,et al.  Netlab: Algorithms for Pattern Recognition , 2002 .

[18]  Martin Schlang,et al.  Neural computation in steel industry , 1999, 1999 European Control Conference (ECC).

[19]  Frank Klawonn,et al.  POLARMAP - a New Approach to Visualisation of High Dimensional Data , 2006 .

[20]  Ron Kohavi,et al.  Wrappers for performance enhancement and oblivious decision graphs , 1995 .

[21]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[22]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[23]  Huan Liu,et al.  Feature Transformation and Multivariate Decision Tree Induction , 1998, Discovery Science.

[24]  Frank Klawonn,et al.  Improved fuzzy partitions for fuzzy regression models , 2003, Int. J. Approx. Reason..

[25]  R. Tibshirani,et al.  Generalized Additive Models , 1991 .

[26]  Paulo J,et al.  Industrial use of safety-related artificial neural networks , 2001 .

[27]  Jim Austin,et al.  Developing artificial neural networks for safety critical systems , 2006, Neural Computing and Applications.

[28]  Helge Ritter,et al.  Learning with the Self-Organizing Map , 1991 .

[29]  Robert P. W. Duin,et al.  Using two-class classifiers for multiclass classification , 2002, Object recognition supported by user interaction for service robots.

[30]  D. Mackay,et al.  Introduction to Gaussian processes , 1998 .

[31]  R. R. Zakrzewski Verification of a trained neural network accuracy , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[32]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[33]  Nancy G. Leveson,et al.  An investigation of the Therac-25 accidents , 1993, Computer.

[34]  Tom Fawcett,et al.  Robust Classification for Imprecise Environments , 2000, Machine Learning.

[35]  Rudolf Kruse,et al.  Learning Verifiable Ensembles for Classification Problems with High Safety Requirements , 2010 .

[36]  Werner Hauptmann,et al.  Verifiable Ensembles of Low-Dimensional Submodels for Multi-class Problems with Imbalanced Misclassification Costs , 2009, Applications of Supervised and Unsupervised Ensemble Methods.

[37]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[38]  R.J. Hathaway,et al.  Switching regression models and fuzzy clustering , 1993, IEEE Trans. Fuzzy Syst..

[39]  C. J. Stone,et al.  Additive Regression and Other Nonparametric Models , 1985 .

[40]  Jonathan P. Bowen,et al.  System and Software Safety in Critical Systems , 1996 .

[41]  Rudolf Kruse,et al.  Maschinelles Lernen von validierbaren Klassifikatoren zur autonomen Steuerung sicherheitsrelevanter SystemeMachine Learning of Verifiable Classifiers for Autonomous Control of Safety-related Systems , 2009, Autom..

[42]  James D. Keeler,et al.  Layered Neural Networks with Gaussian Hidden Units as Universal Approximations , 1990, Neural Computation.

[43]  Sheng Chen,et al.  A Kernel-Based Two-Class Classifier for Imbalanced Data Sets , 2007, IEEE Transactions on Neural Networks.

[44]  Sebastian Thrun,et al.  The MONK''s Problems-A Performance Comparison of Different Learning Algorithms, CMU-CS-91-197, Sch , 1991 .

[45]  Saso Dzeroski,et al.  Inductive Logic Programming: Techniques and Applications , 1993 .

[46]  Claus Weihs,et al.  Local Modelling in Classification on Different Feature Subspaces , 2006, ICDM.

[47]  Werner Hauptmann,et al.  Interpretable ensembles of local models for safety-related applications , 2008, ESANN.

[48]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[49]  Laura L. Pullum,et al.  Guidance for the Verification and Validation of Neural Networks (Emerging Technologies) , 2007 .

[50]  Brian J. Taylor,et al.  Methods and Procedures for the Verification and Validation of Artificial Neural Networks , 2005 .

[51]  Detlef D. Nauck Fuzzy data analysis with NEFCLASS , 2003, Int. J. Approx. Reason..

[52]  J. Susan Milton,et al.  Introduction to Probability and Statistics: Principles and Applications for Engineering and the Computing Sciences , 1990 .

[53]  I-Cheng Yeh,et al.  Modeling of strength of high-performance concrete using artificial neural networks , 1998 .

[54]  Michael Margaliot,et al.  Are artificial neural networks white boxes? , 2005, IEEE Transactions on Neural Networks.

[55]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[56]  D. Mackay,et al.  Bayesian methods for adaptive models , 1992 .

[57]  Werner Hauptmann,et al.  Multi-Class Extension of Verifiable Ensemble Models for Safety-Related Applications , 2008, GfKl.

[58]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..