The superior fault tolerance of artificial neural network training with a fault/noise injection-based genetic algorithm

Artificial neural networks (ANNs) are powerful computational tools that are designed to replicate the human brain and adopted to solve a variety of problems in many different fields. Fault tolerance (FT), an important property of ANNs, ensures their reliability when significant portions of a network are lost. In this paper, a fault/noise injection-based (FIB) genetic algorithm (GA) is proposed to construct fault-tolerant ANNs. The FT performance of an FIB-GA was compared with that of a common genetic algorithm, the back-propagation algorithm, and the modification of weights algorithm. The FIB-GA showed a slower fitting speed when solving the exclusive OR (XOR) problem and the overlapping classification problem, but it significantly reduced the errors in cases of single or multiple faults in ANN weights or nodes. Further analysis revealed that the fit weights showed no correlation with the fitting errors in the ANNs constructed with the FIB-GA, suggesting a relatively even distribution of the various fitting parameters. In contrast, the output weights in the training of ANNs implemented with the use the other three algorithms demonstrated a positive correlation with the errors. Our findings therefore indicate that a combination of the fault/noise injection-based method and a GA is capable of introducing FT to ANNs and imply that the distributed ANNs demonstrate superior FT performance.

[1]  Salvatore Cavalieri,et al.  A novel learning algorithm which improves the partial fault tolerance of multilayer neural networks , 1999, Neural Networks.

[2]  John Maddox,et al.  Genetics helping molecular dynamics , 1995, Nature.

[3]  Steven Hampson,et al.  Generalization and specialization in artificial neural networks , 1991, Progress in Neurobiology.

[4]  Andrew Chi-Sing Leung,et al.  Prediction error of a fault tolerant neural network , 2008, Neurocomputing.

[5]  Dhananjay S. Phatak,et al.  Investigating the Fault Tolerance of Neural Networks , 2005, Neural Computation.

[6]  P. Willett.  Genetic algorithms in molecular recognition and design. , 1995, Trends in biotechnology.

[7]  Andrew Chi-Sing Leung,et al.  Regularizers for fault tolerant multilayer feedforward networks , 2011, Neurocomputing.

[8]  Riccardo Leardi,et al.  Genetic algorithms in chemistry. , 2007, Journal of chromatography. A.

[9]  P. Rajan,et al.  Artificial neural networks in urolithiasis , 2005, Current opinion in urology.

[10]  Dhananjay S. Phatak,et al.  Complete and partial fault tolerance of feedforward neural nets , 1995, IEEE Trans. Neural Networks.

[11]  Paulo J. G. Lisboa,et al.  The Use of Artificial Neural Networks in Decision Support in Cancer: a Systematic Review , 2005 .

[12]  Massimo Buscema,et al.  Artificial Neural Networks, and Evolutionary Algorithms as a systems biology approach to a data-base on fetal growth restriction. , 2013, Progress in biophysics and molecular biology.

[13]  Mo Jamshidi,et al.  Tools for intelligent control: fuzzy controllers, neural networks and genetic algorithms , 2003, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[14]  P. Bosco,et al.  Brain atrophy in Alzheimer’s Disease and aging , 2016, Ageing Research Reviews.

[15]  João L Vilaça,et al.  Automatic modeling of pectus excavatum corrective prosthesis using artificial neural networks. , 2014, Medical engineering & physics.

[16]  Alexander R. A. Anderson,et al.  Evolving Homeostatic Tissue Using Genetic Algorithms , 2011, ALIFE.

[17]  Paul M. Thompson,et al.  A Focus on Structural Brain Imaging in the Alzheimer’s Disease Neuroimaging Initiative , 2014, Biological Psychiatry.

[18]  Elmer Fernandez,et al.  Comparison of algorithms to infer genetic population structure from unlinked molecular markers , 2014, Statistical applications in genetics and molecular biology.

[19]  Franz Rothlauf,et al.  Network Random Keys-A Tree Representation Scheme for Genetic and Evolutionary Algorithms , 2005 .

[20]  K. J. Dalton,et al.  Artificial neural networks for decision support in clinical medicine. , 1995, Annals of medicine.

[21]  R. Dybowski,et al.  Artificial neural networks in pathology and medical laboratories , 1995, The Lancet.

[22]  Mark E. Schmidt,et al.  The Alzheimer's Disease Neuroimaging Initiative: A review of papers published since its inception , 2012, Alzheimer's & Dementia.

[23]  I A Basheer,et al.  Artificial neural networks: fundamentals, computing, design, and application. , 2000, Journal of microbiological methods.

[24]  R Kamimura,et al.  Knowledge-based systems, artificial neural networks and pattern recognition: applications to biotechnological processes. , 1996, Current opinion in biotechnology.

[25]  A. Krogh.  What are artificial neural networks? , 2008, Nature Biotechnology.

[26]  Robert I. Damper,et al.  Determining and improving the fault tolerance of multilayer perceptrons in a pattern-recognition application , 1993, IEEE Trans. Neural Networks.

[27]  S. Hampson.  Problem solving in artificial neural networks , 1994, Progress in Neurobiology.

[28]  Andrew Chi-Sing Leung,et al.  Convergence and Objective Functions of Some Fault/Noise-Injection-Based Online Learning Algorithms for RBF Networks , 2010, IEEE Transactions on Neural Networks.

[29]  Paulo J. G. Lisboa,et al.  A review of evidence of health benefit from artificial neural networks in medical intervention , 2002, Neural Networks.

[30]  Caro Lucas,et al.  Relaxed Fault-Tolerant Hardware Implementation of Neural Networks in the Presence of Multiple Transient Errors , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Andrew Chi-Sing Leung,et al.  Prediction error of a fault tolerant neural network , 2006, Neurocomputing.

[32]  W. Baxt.  Application of artificial neural networks to clinical medicine , 1995, The Lancet.

[33]  Jigneshkumar L Patel,et al.  Applications of artificial neural networks in medical science. , 2007, Current clinical pharmacology.

[34]  L. Weber,et al.  Applications of genetic algorithms in molecular diversity. , 1998, Current opinion in chemical biology.

[35]  J Moult,et al.  Genetic algorithms for protein structure prediction. , 1996, Current opinion in structural biology.

[36]  Ali Montazeri,et al.  Artificial neural networks in neurosurgery , 2014, Journal of Neurology, Neurosurgery & Psychiatry.

[37]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[38]  Chunyun Xu,et al.  Optimization Analysis of Dynamic Sample Number and Hidden Layer Node Number Based on BP Neural Network , 2013, BIC-TA.

[39]  S Forrest,et al.  Genetic algorithms , 1996, CSUR.

[40]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[41]  David A. Medler,et al.  Training redundant artificial neural networks: Imposing biology on technology , 1994, Psychological research.

[42]  Antonio Francipane,et al.  Strategies investigation in using artificial neural network for landslide susceptibility mapping: application to a Sicilian catchment , 2014 .

[43]  Michael W. Weiner,et al.  2014 Update of the Alzheimer's Disease Neuroimaging Initiative: A review of papers published since its inception , 2015, Alzheimer's & Dementia.

[44]  Alan H B Wu,et al.  Use of genetic and nongenetic factors in warfarin dosing algorithms. , 2007, Pharmacogenomics.

[45]  Kurt Miller,et al.  Artificial neural networks and prostate cancer—tools for diagnosis and management , 2013, Nature Reviews Urology.

[46]  Junjie Wu,et al.  Classification with Class Overlapping: A Systematic Study , 2010 .

[47]  Andrew Chi-Sing Leung,et al.  A Fault-Tolerant Regularizer for RBF Networks , 2008, IEEE Transactions on Neural Networks.

[48]  Brian C. Lovell,et al.  The Multiscale Classifier , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Jun Sawamoto,et al.  Neural Network to Control Output of Hidden Node According to Input Patterns , 2014 .

[50]  Rui Huang,et al.  A meta-analysis of voxel-based morphometry studies of white matter volume alterations in Alzheimer's disease , 2012, Neuroscience & Biobehavioral Reviews.

[51]  Ricard V Solé,et al.  Distributed robustness in cellular networks: insights from synthetic evolved circuits , 2009, Journal of The Royal Society Interface.

[52]  Gee Wah Ng,et al.  Classification for overlapping classes using optimized overlapping region detection and soft decision , 2010, 2010 13th International Conference on Information Fusion.

[53]  F Liu,et al.  [Genetic algorithms and its application to spectral analysis]. , 2001, Guang pu xue yu guang pu fen xi = Guang pu.

[54]  N. Fayed,et al.  Magnetic resonance imaging based clinical research in Alzheimer's disease. , 2012, Journal of Alzheimer's disease : JAD.

[55]  Daniel L. Palumbo,et al.  Performance and fault-tolerance of neural networks for optimization , 1993, IEEE Trans. Neural Networks.

[56]  Scott R. Presnell,et al.  Artificial neural networks for pattern recognition in biochemical sequences. , 1993, Annual review of biophysics and biomolecular structure.

[57]  Dingde Jiang,et al.  How to reconstruct end-to-end traffic based on time-frequency analysis and artificial neural network , 2014 .

[58]  Jonas S Almeida,et al.  Predictive non-linear modeling of complex data by artificial neural networks. , 2002, Current opinion in biotechnology.

[59]  N Meurice,et al.  Comparison of benzodiazepine-like compounds using topological analysis and genetic algorithms. , 1998, SAR and QSAR in environmental research.