Generalized type-2 fuzzy weight adjustment for backpropagation neural networks in time series prediction

In this paper the comparison of a proposed neural network with generalized type-2 fuzzy weights (NNGT2FW) with respect to the monolithic neural network (NN) and the neural network with interval type-2 fuzzy weights (NNIT2FW) is presented. Generalized type-2 fuzzy inference systems are used to obtain the generalized type-2 fuzzy weights and are designed by a strategy of increasing and decreasing an epsilon variable for obtaining the different sizes of the footprint of uncertainty (FOU) for the generalized membership functions. The proposed method is based on recent approaches that handle weight adaptation using type-1 and type-2 fuzzy logic. The approach is applied to the prediction of the Mackey-Glass time series, and results are shown to outperform the results produced by other neural models. Gaussian noise was applied to the test data of the Mackey-Glass time series for finding out which of the presented methods in this paper shows better performance and tolerance to noise.

[1]  Cuntai Guan,et al.  eT2FIS: An Evolving Type-2 Neural Fuzzy Inference System , 2013, Inf. Sci..

[2]  Jerry M. Mendel,et al.  Type-2 fuzzy sets made simple , 2002, IEEE Trans. Fuzzy Syst..

[3]  Sorin Draghici,et al.  On the capabilities of neural networks using limited precision weights , 2002, Neural Networks.

[4]  Oscar Castillo,et al.  A review on the design and optimization of interval type-2 fuzzy controllers , 2012, Appl. Soft Comput..

[5]  Dongrui Wu,et al.  Approaches for Reducing the Computational Cost of Interval Type-2 Fuzzy Logic Systems: Overview and Comparisons , 2013, IEEE Transactions on Fuzzy Systems.

[6]  Jerry M. Mendel,et al.  Comments on "alpha -Plane Representation for Type-2 Fuzzy Sets: Theory and Applications" , 2010, IEEE Trans. Fuzzy Syst..

[7]  Tomás Gedeon,et al.  Additive neural networks and periodic patterns , 1999, Neural Networks.

[8]  Oscar Castillo,et al.  Genetic optimization of ensemble neural networks for complex time series prediction , 2011, The 2011 International Joint Conference on Neural Networks.

[9]  Patrick P. K. Chan,et al.  Radial Basis Function network learning using localized generalization error bound , 2009, Inf. Sci..

[10]  Mohammad Hossein Fazel Zarandi,et al.  On type-reduction of type-2 fuzzy sets: A review , 2015, Appl. Soft Comput..

[11]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[12]  P. S. Sastry,et al.  Analysis of the back-propagation algorithm with momentum , 1994, IEEE Trans. Neural Networks.

[13]  Tommy W. S. Chow,et al.  A weight initialization method for improving training speed in feedforward neural network , 2000, Neurocomputing.

[14]  Oscar Castillo,et al.  A New Biometric Recognition Technique Based on Hand Geometry and Voice Using Neural Networks and Fuzzy Logic , 2008, Soft Computing for Hybrid Intelligent Systems.

[15]  Oscar Castillo,et al.  An Efficient Computational Method to Implement Type-2 Fuzzy Logic in Control Applications , 2007, Analysis and Design of Intelligent Systems using Soft Computing Techniques.

[16]  Jerry M. Mendel,et al.  $\alpha$-Plane Representation for Type-2 Fuzzy Sets: Theory and Applications , 2009, IEEE Transactions on Fuzzy Systems.

[17]  Oscar Castillo,et al.  An optimization method for designing type-2 fuzzy inference systems based on the footprint of uncertainty using genetic algorithms , 2012, Expert Syst. Appl..

[18]  Philippe De Wilde.  The Magnitude of the Diagonal Elements in Neural Networks , 1997, Neural Networks.

[19]  Ioannis B. Theocharis,et al.  Locally recurrent neural networks for wind speed prediction using spatial correlation , 2007, Inf. Sci..

[20]  Myriam Regattieri Delgado,et al.  General Type-2 Fuzzy Inference Systems: Analysis, Design and Computational Aspects , 2007, 2007 IEEE International Fuzzy Systems Conference.

[21]  Francisco Chiclana,et al.  Defuzzification of the discretised generalised type-2 fuzzy set: Experimental evaluation , 2013, Inf. Sci..

[22]  Richard Stuart Neville,et al.  Transformations of sigma-pi nets: obtaining reflected functions by reflecting weight matrices , 2002, Neural Networks.

[23]  Patricia Melin,et al.  Modular Neural Networks and Type-2 Fuzzy Systems for Pattern Recognition , 2011, Studies in Computational Intelligence.

[24]  Jerry M. Mendel,et al.  On KM Algorithms for Solving Type-2 Fuzzy Set Problems , 2013, IEEE Transactions on Fuzzy Systems.

[25]  Hisao Ishibuchi,et al.  Fuzzy neural networks with fuzzy weights and fuzzy biases , 1993, IEEE International Conference on Neural Networks.

[26]  E. Mizutani,et al.  Neuro-Fuzzy and Soft Computing-A Computational Approach to Learning and Machine Intelligence [Book Review] , 1997, IEEE Transactions on Automatic Control.

[27]  Oscar Castillo,et al.  Building Fuzzy Inference Systems with a New Interval Type-2 Fuzzy Logic Toolbox , 2007, Trans. Comput. Sci..

[28]  Jerry M. Mendel,et al.  Type-2 fuzzy logic systems , 1999, IEEE Trans. Fuzzy Syst..

[29]  Witold Pedrycz,et al.  Type-2 fuzzy neural networks with fuzzy clustering and differential evolution optimization , 2011, Inf. Sci..

[30]  Jerry M. Mendel,et al.  Uncertainty measures for general type-2 fuzzy sets , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[31]  Mohammad Hossein Fazel Zarandi,et al.  A new indirect approach to the type-2 fuzzy systems modeling and design , 2013, Inf. Sci..

[32]  Héctor Pomares,et al.  Multiobjective Optimization and Comparison of Nonsingleton Type-1 and Singleton Interval Type-2 Fuzzy Logic Systems , 2013, IEEE Transactions on Fuzzy Systems.

[33]  Oscar Castillo,et al.  A hybrid learning algorithm for a class of interval type-2 fuzzy neural networks , 2009, Inf. Sci..

[34]  Kazuyuki Murase,et al.  A new algorithm to design compact two-hidden-layer artificial neural networks , 2001, Neural Networks.

[35]  J. Mendel.  Uncertain Rule-Based Fuzzy Logic Systems: Introduction and New Directions , 2001 .

[36]  Min Zhou,et al.  Footprint of uncertainty for type-2 fuzzy sets , 2014, Inf. Sci..

[37]  M. J. D. Powell,et al.  Restart procedures for the conjugate gradient method , 1977, Math. Program..

[38]  Jerry M. Mendel,et al.  Applications of Type-2 Fuzzy Logic Systems to Forecasting of Time-series , 1999, Inf. Sci..

[39]  Feilong Liu,et al.  An efficient centroid type-reduction strategy for general type-2 fuzzy logic system , 2008, Inf. Sci..

[40]  Miguel Cazorla,et al.  Two Bayesian methods for junction classification , 2003, IEEE Trans. Image Process..

[41]  Patricia Melin,et al.  Neural Network with Fuzzy Weights Using Type-1 and Type-2 Fuzzy Learning with Gaussian Membership Functions , 2014, Recent Advances on Hybrid Approaches for Designing Intelligent Systems.

[42]  Patricia Melin,et al.  Modular Neural Networks and Fuzzy Sugeno Integral for Face and Fingerprint Recognition , 2004, WSC.

[43]  Oscar Castillo,et al.  An Interval Type-2 Fuzzy Logic Toolbox for Control Applications , 2007, 2007 IEEE International Fuzzy Systems Conference.

[44]  T. Feuring,et al.  Learning in fuzzy neural networks , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[45]  Witold Pedrycz,et al.  Granular Computing: Analysis and Design of Intelligent Systems , 2013 .

[46]  David Casasent,et al.  A classifier neural net with complex-valued weights and square-law nonlinearities , 1995, Neural Networks.

[47]  J. Mendel,et al.  Centroid of a general type-2 fuzzy set computed by means of the centroid-flow algorithm , 2010, International Conference on Fuzzy Systems.

[48]  Oscar Castillo,et al.  Interval type-2 fuzzy weight adjustment for backpropagation neural networks with application in time series prediction , 2014, Inf. Sci..

[49]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[50]  Hani Hagras,et al.  Type-2 Fuzzy Logic Controllers: A Way Forward for Fuzzy Systems in Real World Environments , 2008, WCCI.

[51]  Oscar Castillo,et al.  An Interval Type-2 Fuzzy Neural Network for Chaotic Time Series Prediction with Cross-Validation and Akaike Test , 2011, Soft Computing for Intelligent Control and Mobile Robotics.

[52]  Moshe Shoham,et al.  Approximating Functions by Neural Networks: A Constructive Solution in the Uniform Norm , 1996, Neural Networks.

[53]  Sagar V. Kamarthi,et al.  Accelerating neural network training using weight extrapolations , 1999, Neural Networks.

[54]  Hideo Tanaka,et al.  A fuzzy neural network with trapezoid fuzzy weights , 1994, Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference.

[55]  Rahib Hidayat Abiyev,et al.  A Type-2 Fuzzy Wavelet Neural Network for Time Series Prediction , 2010, IEA/AIE.

[56]  Hani Hagras,et al.  Toward General Type-2 Fuzzy Logic Systems Based on zSlices , 2010, IEEE Transactions on Fuzzy Systems.

[57]  Jerry M. Mendel,et al.  Study on enhanced Karnik-Mendel algorithms: Initialization explanations and computation improvements , 2012, Inf. Sci..