Relaxed Fault-Tolerant Hardware Implementation of Neural Networks in the Presence of Multiple Transient Errors

Reliability should be identified as the most important challenge in future nano-scale very large scale integration (VLSI) implementation technologies for the development of complex integrated systems. Normally, fault tolerance (FT) in a conventional system is achieved by increasing its redundancy, which also implies higher implementation costs and lower performance that sometimes makes it even infeasible. In contrast to custom approaches, a new class of applications is categorized in this paper, which is inherently capable of absorbing some degrees of vulnerability and providing FT based on their natural properties. Neural networks are good indicators of imprecision-tolerant applications. We have also proposed a new class of FT techniques called relaxed fault-tolerant (RFT) techniques which are developed for VLSI implementation of imprecision-tolerant applications. The main advantage of RFT techniques with respect to traditional FT solutions is that they exploit inherent FT of different applications to reduce their implementation costs while improving their performance. To show the applicability as well as the efficiency of the RFT method, the experimental results for implementation of a face-recognition computationally intensive neural network and its corresponding RFT realization are presented in this paper. The results demonstrate promising higher performance of artificial neural network VLSI solutions for complex applications in faulty nano-scale implementation environments.

[1]  Kaushik Roy,et al.  Process Variation Tolerant Low Power DCT Architecture , 2007, 2007 Design, Automation & Test in Europe Conference & Exhibition.

[2]  Elaheh Bozorgzadeh,et al.  Single-Event-Upset (SEU) Awareness in FPGA Routing , 2007, 2007 44th ACM/IEEE Design Automation Conference.

[3]  Sandip Kundu,et al.  A Built-In Self-Test Scheme for Soft Error Rate Characterization , 2008, 2008 14th IEEE International On-Line Testing Symposium.

[4]  Wei Qiao,et al.  Fault-Tolerant Indirect Adaptive Neurocontrol for a Static Synchronous Series Compensator in a Power Network With Missing Sensor Measurements , 2008, IEEE Transactions on Neural Networks.

[5]  Zheng Li Fault diagnosis and fault tolerant control of mobile robot based on neural networks , 2009, 2009 International Conference on Machine Learning and Cybernetics.

[6]  D. Merodio,et al.  Experimental Validation of a Tool for Predicting the Effects of Soft Errors in SRAM-Based FPGAs , 2007, IEEE Transactions on Nuclear Science.

[7]  John P. Hayes,et al.  An Analysis Framework for Transient-Error Tolerance , 2007, 25th IEEE VLSI Test Symposium (VTS'07).

[8]  T.M. Mak,et al.  Built-In Soft Error Resilience for Robust System Design , 2007, 2007 IEEE International Conference on Integrated Circuit Design and Technology.

[9]  Masayuki Murakami,et al.  Fault Tolerance Comparison of IDS Models with Multilayer Perceptron and Radial Basis Function Networks , 2007, 2007 International Joint Conference on Neural Networks.

[10]  Alan F. Murray,et al.  Large Developing Receptive Fields Using a Distributed and Locally Reprogrammable Address–Event Receiver , 2010, IEEE Transactions on Neural Networks.

[11]  Johan Karlsson,et al.  Software Implemented Detection and Recovery of Soft Errors in a Brake-by-Wire System , 2008, 2008 Seventh European Dependable Computing Conference.

[12]  Andrew Hunter,et al.  A modified sparse distributed memory model for extracting clean patterns from noisy inputs , 2009, 2009 International Joint Conference on Neural Networks.

[13]  Bronis R. de Supinski,et al.  Soft error vulnerability of iterative linear algebra methods , 2007, ICS '08.

[14]  Yan Lin,et al.  Device and architecture concurrent optimization for FPGA transient soft error rate , 2007, 2007 IEEE/ACM International Conference on Computer-Aided Design.

[15]  Weidong Kuang,et al.  Soft digital signal processing using self-timed circuits , 2002, ICONIP '02. Proceedings of the 9th International Conference on Neural Information Processing. Computational Intelligence for the E-Age (IEEE Cat. No.02EX575).

[16]  Yogesh Singh,et al.  Fault tolerance of feedforward artificial neural networks- a framework of study , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[17]  Dhiraj K. Pradhan,et al.  Fault tolerant bit parallel finite field multipliers using LDPC codes , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[18]  Javier Echanobe,et al.  Efficient Hardware/Software Implementation of an Adaptive Neuro-Fuzzy System , 2008, IEEE Transactions on Fuzzy Systems.

[19]  Tobi Delbrück,et al.  CAVIAR: A 45k Neuron, 5M Synapse, 12G Connects/s AER Hardware Sensory–Processing– Learning–Actuating System for High-Speed Visual Object Recognition and Tracking , 2009, IEEE Transactions on Neural Networks.

[20]  Ramesh Karri,et al.  Logic Level Fault Tolerance Approaches Targeting Nanoelectronics PLAs , 2007, 2007 Design, Automation & Test in Europe Conference & Exhibition.

[21]  Bernabe Linares-Barranco,et al.  Comparison between Frame-Constrained Fix-Pixel-Value and Frame-Free Spiking-Dynamic-Pixel ConvNets for Visual Processing , 2012, Front. Neurosci..

[22]  Emile Fiesler,et al.  Neural Network Adaptations to Hardware Implementations , 1997 .

[23]  Sotirios Matakias,et al.  A Circuit for Concurrent Detection of Soft and Timing Errors in Digital CMOS ICs , 2004, J. Electron. Test..

[24]  Pedro Reviriego,et al.  Protection against soft errors in the space environment: A finite impulse response (FIR) filter case study , 2009, Integr..

[25]  David Walker,et al.  Fault-tolerant typed assembly language , 2007, PLDI '07.

[26]  Tülay Yildirim,et al.  Conic Section Function Neural Network Circuitry for Offline Signature Recognition , 2010, IEEE Transactions on Neural Networks.

[27]  Melvin A. Breuer,et al.  Multi-media applications and imprecise computation , 2005, 8th Euromicro Conference on Digital System Design (DSD'05).

[28]  Giacomo Indiveri,et al.  Real-Time Classification of Complex Patterns Using Spike-Based Learning in Neuromorphic VLSI , 2009, IEEE Transactions on Biomedical Circuits and Systems.

[29]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[30]  Dharmendra S. Modha,et al.  A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[31]  Wen-Shyong Yu,et al.  Finite wordlength effects analysis of a digital neural adaptive tracking controller for a class of nonlinear dynamical systems , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[32]  J. Tombs,et al.  A non-invasive system for the measurement of the robustness of microprocessor-type architectures against radiation-induced soft errors , 2008, 2008 IEEE Instrumentation and Measurement Technology Conference.

[33]  Stamatis Vassiliadis,et al.  Instruction-Level Fault Tolerance Configurability , 2007, 2007 International Conference on Embedded Computer Systems: Architectures, Modeling and Simulation.

[34]  Luigi Carro,et al.  Functionally Fault-tolerant DSP Microprocessor using Sigma–delta Modulated Signals , 2007, J. Electron. Test..

[35]  P. Reviriego,et al.  Efficient Protection Techniques Against SEUs for Adaptive Filters: An Echo Canceller Case Study , 2008, IEEE Transactions on Nuclear Science.

[36]  Craig B. Zilles,et al.  A characterization of instruction-level error derating and its implications for error detection , 2008, 2008 IEEE International Conference on Dependable Systems and Networks With FTCS and DCC (DSN).

[37]  Benjamin W. Wah,et al.  Fault tolerant neural networks with hybrid redundancy , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[38]  Raoul Velazco,et al.  SEU fault tolerance in artificial neural networks , 1995 .

[39]  Luigi Carro,et al.  System Level Approaches for Mitigation of Long Duration Transient Faults in Future Technologies , 2007, 12th IEEE European Test Symposium (ETS'07).

[40]  Muhammad Sheikh Sadi,et al.  A Design Approach for Soft Error Protection in Real-Time Embedded Systems , 2008, 19th Australian Conference on Software Engineering (aswec 2008).

[41]  Hyuk-Jun Lee Immediate soft error detection using pass gate logic for content addressable memory , 2008 .

[42]  Michael F. P. O'Boyle,et al.  Evaluating the Effects of Compiler Optimisations on AVF , 2008 .

[43]  Johannes Schemmel,et al.  Realizing biological spiking network models in a configurable wafer-scale hardware system , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[44]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[45]  David H. K. Hoe,et al.  Fault tolerant Block Based Neural Networks , 2010, 2010 42nd Southeastern Symposium on System Theory (SSST).

[46]  Todd M. Austin,et al.  A Systematic Methodology to Compute the Architectural Vulnerability Factors for a High-Performance Microprocessor , 2003, MICRO.

[47]  Narayanan Vijaykrishnan,et al.  Modeling Soft Errors at the Device and Logic Levels for Combinational Circuits , 2009, IEEE Transactions on Dependable and Secure Computing.

[48]  Alessandro Artusi,et al.  Radial Basis Function Networks GPU-Based Implementation , 2008, IEEE Transactions on Neural Networks.

[49]  Bernard Girau,et al.  Hardware implementation of Spiking Neural Network classifiers based on backpropagation-based learning algorithms , 2009, 2009 International Joint Conference on Neural Networks.

[50]  Mehdi Baradaran Tahoori,et al.  Analytical Techniques for Soft Error Rate Modeling and Mitigation of FPGA-Based Designs , 2007, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[51]  Bernabé Linares-Barranco,et al.  Fast Vision Through Frameless Event-Based Sensing and Convolutional Processing: Application to Texture Recognition , 2010, IEEE Transactions on Neural Networks.

[52]  E. Kidwell Brainchild: a fault tolerant neural network , 1989, International 1989 Joint Conference on Neural Networks.

[53]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[54]  Andrew Chi-Sing Leung,et al.  Convergence and Objective Functions of Some Fault/Noise-Injection-Based Online Learning Algorithms for RBF Networks , 2010, IEEE Transactions on Neural Networks.

[55]  Caro Lucas,et al.  Bio-Inspired Imprecise Computational Blocks for Efficient VLSI Implementation of Soft-Computing Applications , 2010, IEEE Transactions on Circuits and Systems I: Regular Papers.

[56]  Marcian N. Cirstea,et al.  Direct Neural-Network Hardware-Implementation Algorithm , 2010, IEEE Transactions on Industrial Electronics.

[57]  Reto Zimmermann,et al.  Binary adder architectures for cell-based VLSI and their synthesis , 1997 .

[58]  Diana Marculescu,et al.  Soft error rate reduction using redundancy addition and removal , 2008, 2008 Asia and South Pacific Design Automation Conference.

[59]  David Harris,et al.  CMOS VLSI Design: A Circuits and Systems Perspective , 2004 .

[60]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[61]  Yung-Yuan Chen,et al.  An Estimation Model of Vulnerability for Embedded Microprocessors , 2008, 2008 Second International Conference on Secure System Integration and Reliability Improvement.

[62]  Nihar R. Mahapatra,et al.  Energy-Efficient Soft-Error Protection Using Operand Encoding and Operation Bypass , 2008, 21st International Conference on VLSI Design (VLSID 2008).

[63]  Subhasish Mitra Globally Optimized Robust Systems to Overcome Scaled CMOS Reliability Challenges , 2008, 2008 Design, Automation and Test in Europe.

[64]  Paul Chow,et al.  High-Performance Reconfigurable Hardware Architecture for Restricted Boltzmann Machines , 2010, IEEE Transactions on Neural Networks.

[65]  Stephen P. DeWeerth,et al.  An Integrated System for Simultaneous, Multichannel Neuronal Stimulation and Recording , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[66]  Hans-Joachim Wunderlich,et al.  Integrating Scan Design and Soft Error Correction in Low-Power Applications , 2008, 2008 14th IEEE International On-Line Testing Symposium.

[67]  Nam Ik Cho,et al.  Low-Power Filtering Via Minimum Power Soft Error Cancellation , 2007, IEEE Transactions on Signal Processing.

[68]  David J. Lilja,et al.  Scaling Analytical Models for Soft Error Rate Estimation Under a Multiple-Fault Environment , 2007, 10th Euromicro Conference on Digital System Design Architectures, Methods and Tools (DSD 2007).