Genetic Optimization of Type-2 Fuzzy Weight Adjustment for Backpropagation in Ensemble Neural Network

In this paper a genetic algorithm is used to optimize the three neural networks forming an ensemble. Genetic algorithms are also used to optimize the two type-2 fuzzy systems that work in the backpropagation learning method with type-2 fuzzy weight adjustment. The mathematical analysis of the proposed learning method architecture and the adaptation of type-2 fuzzy weights are presented. The proposed method is based on recent methods that handle weight adaptation and especially fuzzy weights. In this work an ensemble neural network of three neural networks and average integration to obtain the final result is presented. The proposed approach is applied to a case of time series prediction.

[1]  Patricia Melin,et al.  Modular Neural Networks and Fuzzy Sugeno Integral for Face and Fingerprint Recognition , 2004, WSC.

[2]  T. Feuring,et al.  Learning in fuzzy neural networks , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[3]  M. J. D. Powell,et al.  Restart procedures for the conjugate gradient method , 1977, Math. Program..

[4]  Rahib Hidayat Abiyev,et al.  A Type-2 Fuzzy Wavelet Neural Network for Time Series Prediction , 2010, IEA/AIE.

[5]  Nicolás García-Pedrajas,et al.  Trends in Applied Intelligent Systems - 23rd International Conference on Industrial Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2010, Cordoba, Spain, June 1-4, 2010, Proceedings, Part I , 2010, IEA/AIE.

[6]  Jerry M. Mendel,et al.  Applications of Type-2 Fuzzy Logic Systems to Forecasting of Time-series , 1999, Inf. Sci..

[7]  Patricia Melin,et al.  Modular Neural Network with Fuzzy Integration and Its Optimization Using Genetic Algorithms for Human Recognition Based on Iris, Ear and Voice Biometrics , 2010, Soft Computing for Recognition Based on Biometrics.

[8]  David Casasent,et al.  A classifier neural net with complex-valued weights and square-law nonlinearities , 1995, Neural Networks.

[9]  Miguel Cazorla,et al.  Two Bayesian methods for junction classification , 2003, IEEE Trans. Image Process..

[10]  P. S. Sastry,et al.  Analysis of the back-propagation algorithm with momentum , 1994, IEEE Trans. Neural Networks.

[11]  Oscar Castillo,et al.  A New Biometric Recognition Technique Based on Hand Geometry and Voice Using Neural Networks and Fuzzy Logic , 2008, Soft Computing for Hybrid Intelligent Systems.

[12]  Oscar Castillo,et al.  An Efficient Computational Method to Implement Type-2 Fuzzy Logic in Control Applications , 2007, Analysis and Design of Intelligent Systems using Soft Computing Techniques.

[13]  Oscar Castillo,et al.  A review on the design and optimization of interval type-2 fuzzy controllers , 2012, Appl. Soft Comput..

[14]  Richard Stuart Neville,et al.  Transformations of sigma-pi nets: obtaining reflected functions by reflecting weight matrices , 2002, Neural Networks.

[15]  Patricia Melin,et al.  Modular Neural Networks and Type-2 Fuzzy Systems for Pattern Recognition , 2011, Studies in Computational Intelligence.

[16]  Hisao Ishibuchi,et al.  Fuzzy neural networks with fuzzy weights and fuzzy biases , 1993, IEEE International Conference on Neural Networks.

[17]  Sorin Draghici,et al.  On the capabilities of neural networks using limited precision weights , 2002, Neural Networks.

[18]  Patricia Melin,et al.  Parallel genetic algorithms for optimization of Modular Neural Networks in pattern recognition , 2011, The 2011 International Joint Conference on Neural Networks.

[19]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[20]  Kazuyuki Murase,et al.  A new algorithm to design compact two-hidden-layer artificial neural networks , 2001, Neural Networks.

[21]  Hani Hagras,et al.  Type-2 Fuzzy Logic Controllers: A Way Forward for Fuzzy Systems in Real World Environments , 2008, WCCI.

[22]  Tommy W. S. Chow,et al.  A weight initialization method for improving training speed in feedforward neural network , 2000, Neurocomputing.

[23]  Ioannis B. Theocharis,et al.  Locally recurrent neural networks for wind speed prediction using spatial correlation , 2007, Inf. Sci..

[24]  Martin T. Hagan,et al.  Neural network design , 1995 .

[25]  Randy L. Haupt,et al.  Practical Genetic Algorithms , 1998 .

[26]  Oscar Castillo,et al.  Genetic optimization of ensemble neural networks for complex time series prediction , 2011, The 2011 International Joint Conference on Neural Networks.

[27]  Oscar Castillo,et al.  An Interval Type-2 Fuzzy Neural Network for Chaotic Time Series Prediction with Cross-Validation and Akaike Test , 2011, Soft Computing for Intelligent Control and Mobile Robotics.

[28]  Moshe Shoham,et al.  Approximating Functions by Neural Networks: A Constructive Solution in the Uniform Norm , 1996, Neural Networks.

[29]  Sagar V. Kamarthi,et al.  Accelerating neural network training using weight extrapolations , 1999, Neural Networks.

[30]  Hideo Tanaka,et al.  A fuzzy neural network with trapezoid fuzzy weights , 1994, Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference.

[31]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[32]  Oscar Castillo,et al.  A hybrid learning algorithm for a class of interval type-2 fuzzy neural networks , 2009, Inf. Sci..

[33]  Philippe De Wilde.  The Magnitude of the Diagonal Elements in Neural Networks , 1997, Neural Networks.

[34]  E. Mizutani,et al.  Neuro-Fuzzy and Soft Computing-A Computational Approach to Learning and Machine Intelligence [Book Review] , 1997, IEEE Transactions on Automatic Control.

[35]  Tomás Gedeon,et al.  Additive neural networks and periodic patterns , 1999, Neural Networks.

[36]  Patrick P. K. Chan,et al.  Radial Basis Function network learning using localized generalization error bound , 2009, Inf. Sci..