A hybrid cascade neural network with an optimized pool in each cascade

This paper proposes a new architecture and learning algorithms for a hybrid cascade neural network with pool optimization in each cascade. The proposed system is different from existing cascade systems in its capability to operate in an online mode, which allows it to work with non-stationary and stochastic nonlinear chaotic signals with the required accuracy. Compared to conventional analogs, the proposed system provides computational simplicity and possesses both tracking and filtering capabilities.

[1]  Plamen P. Angelov,et al.  Simpl_eTS: a simplified method for learning evolving Takagi-Sugeno fuzzy models , 2005, The 14th IEEE International Conference on Fuzzy Systems, 2005. FUZZ '05..

[2]  Edwin Lughofer,et al.  Improving the robustness of data-driven fuzzy systems with regularization , 2008, 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence).

[3]  E. Mizutani,et al.  Neuro-Fuzzy and Soft Computing-A Computational Approach to Learning and Machine Intelligence [Book Review] , 1997, IEEE Transactions on Automatic Control.

[4]  E. Lughofer,et al.  Model-based fault detection in multi-sensor measurement systems , 2004, 2004 2nd International IEEE Conference on 'Intelligent Systems'. Proceedings (IEEE Cat. No.04EX791).

[5]  Edwin Lughofer,et al.  Evolving Fuzzy Systems - Methodologies, Advanced Concepts and Applications , 2011, Studies in Fuzziness and Soft Computing.

[6]  Plamen P. Angelov,et al.  Data-driven evolving fuzzy systems using eTS and FLEXFIS: comparative analysis , 2008, Int. J. Gen. Syst..

[7]  S. Kaczmarz.  Approximate solution of systems of linear equations , 1993 .

[8]  Ulrich Bodenhofer,et al.  Incremental Learning of Fuzzy Basis Function Networks with a Modified Version of Vector Quantization , 2006 .

[9]  Edwin Lughofer,et al.  On-Line Fault Detection with Data-Driven Evolving Fuzzy Models , 2008, Control. Intell. Syst..

[10]  Visakan Kadirkamanathan,et al.  A Function Estimation Approach to Sequential Learning with Neural Networks , 1993, Neural Computation.

[11]  Plamen Angelov,et al.  On-line Identification of MIMO Evolving Takagi- , 2004 .

[12]  Plamen P. Angelov,et al.  Evolving Fuzzy-Rule-Based Classifiers From Data Streams , 2008, IEEE Transactions on Fuzzy Systems.

[13]  Edwin Lughofer,et al.  Applying evolving fuzzy models with adaptive local error bars to on-line fault detection , 2008, 2008 3rd International Workshop on Genetic and Evolving Systems.

[14]  Edwin Lughofer,et al.  An approach to model-based fault detection in industrial measurement systems with application to engine test benches , 2006 .

[15]  Nikola K. Kasabov,et al.  Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[16]  Edwin Lughofer,et al.  SparseFIS: Data-Driven Learning of Fuzzy Systems With Sparsity Constraints , 2010, IEEE Transactions on Fuzzy Systems.

[17]  Y Lu,et al.  A Sequential Learning Scheme for Function Approximation Using Minimal Radial Basis Function Neural Networks , 1997, Neural Computation.

[18]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[19]  Edwin Lughofer,et al.  Extensions of vector quantization for incremental clustering , 2008, Pattern Recognit..

[20]  Yevgeniy Bodyanskiy,et al.  HYBRID CASCADE NEURAL NETWORK BASED ON WAVELET-NEURON , 2011 .

[21]  E. Lughofer.  Process Safety Enhancements for Data-Driven Evolving Fuzzy Models , 2006, 2006 International Symposium on Evolving Fuzzy Systems.

[22]  D.P. Filev,et al.  An approach to online identification of Takagi-Sugeno fuzzy models , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  Illya Kokshenev,et al.  An adaptive learning algorithm for a neo fuzzy neuron , 2003, EUSFLAT Conf..

[24]  Paramasivan Saratchandran,et al.  Sequential Adaptive Fuzzy Inference System (SAFIS) for nonlinear system identification and prediction , 2006, Fuzzy Sets Syst..

[25]  Frank Fallside,et al.  An adaptive training algorithm for back propagation networks , 1987 .

[26]  Edwin Lughofer,et al.  On-line evolving image classifiers and their application to surface inspection , 2010, Image Vis. Comput..

[27]  Nikola Kasabov,et al.  Evolving connectionist systems , 2002 .

[28]  E. Lughofer,et al.  Evolving fuzzy classifiers using different model architectures , 2008, Fuzzy Sets Syst..

[29]  Yevgeniy V. Bodyanskiy,et al.  A new learning algorithm for a forecasting neuro-fuzzy network , 2003, Integr. Comput. Aided Eng..

[30]  Javier R. Movellan,et al.  Benefits of gain: speeded learning and minimal hidden layers in back-propagation networks , 1991, IEEE Trans. Syst. Man Cybern..

[31]  Nikola K. Kasabov,et al.  DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction , 2002, IEEE Trans. Fuzzy Syst..

[32]  Edwin Lughofer,et al.  On Human-Machine Interaction during Online Image Classifier Training , 2008, 2008 International Conference on Computational Intelligence for Modelling Control & Automation.

[33]  Yevgeniy Bodyanskiy,et al.  EVOLVING CASCADE NEURAL NETWORK BASED ON MULTIDIMESNIONAL EPANECHNIKOV'S KERNELS AND ITS LEARNING ALGORITHM , 2011 .

[34]  Yevgeniy V. Bodyanskiy,et al.  An Adaptive Learning Algorithm for a Neuro-fuzzy Network , 2001, Fuzzy Days.

[35]  Robert J. Schalkoff,et al.  Artificial neural networks , 1997 .

[36]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[37]  Plamen P. Angelov,et al.  Evolving Single- And Multi-Model Fuzzy Classifiers with FLEXFIS-Class , 2007, 2007 IEEE International Fuzzy Systems Conference.

[38]  Yevgeniy Bodyanskiy,et al.  THE CASCADE GROWING NEURAL NETWORK USING QUADRATIC NEURONS AND ITS LEARNING ALGORITHMS FOR ON-LINE INFORMATION PROCESSING , 2009 .

[39]  Plamen P. Angelov,et al.  Architectures for evolving fuzzy rule-based classifiers , 2007, 2007 IEEE International Conference on Systems, Man and Cybernetics.

[40]  N. Kasabov,et al.  Incremental learning in autonomous systems: evolving connectionist systems for on-line image and speech recognition , 2005, IEEE Workshop on Advanced Robotics and its Social Impacts, 2005..

[41]  G. V. Barkan,et al.  CASCADE NEURAL NETWORKS , 1999 .

[43]  Edwin Lughofer,et al.  On Dynamic Selection of the Most Informative Samples in Classification Problems , 2010, 2010 Ninth International Conference on Machine Learning and Applications.

[44]  P. Angelov,et al.  Evolving Fuzzy Systems from Data Streams in Real-Time , 2006, 2006 International Symposium on Evolving Fuzzy Systems.

[45]  Plamen P. Angelov,et al.  Handling drifts and shifts in on-line data streams with evolving fuzzy systems , 2011, Appl. Soft Comput..

[46]  Luís B. Almeida,et al.  Speeding up Backpropagation , 1990 .

[47]  Edwin Lughofer,et al.  Human–Machine Interaction Issues in Quality Control Based on Online Image Classification , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[48]  P. Angelov,et al.  Two approaches to data-driven design of evolving fuzzy systems: eTS and FLEXFIS , 2005, NAFIPS 2005 - 2005 Annual Meeting of the North American Fuzzy Information Processing Society.

[49]  Plamen P. Angelov,et al.  Detecting and Reacting on Drifts and Shifts in On-Line Data Streams with Evolving Fuzzy Systems , 2009, IFSA/EUSFLAT Conf..

[50]  Lutz Prechelt,et al.  Investigation of the CasCor Family of Learning Algorithms , 1997, Neural Networks.

[51]  Takeshi Yamakawa,et al.  Soft Computing Based Signal Prediction, Restoration, and Filtering , 1997 .

[52]  TSUTOMU MIKI.  Analog Implementation of Neo-Fuzzy Neuron and Its On-board Learning , 1999 .

[53]  Edwin Lughofer,et al.  Evolving Vector Quantization for Classification of On-Line Data Streams , 2008, 2008 International Conference on Computational Intelligence for Modelling Control & Automation.

[54]  Edwin Lughofer,et al.  FLEXFIS: A Robust Incremental Learning Approach for Evolving Takagi–Sugeno Fuzzy Models , 2008, IEEE Transactions on Fuzzy Systems.

[55]  Nikola Kasabov,et al.  Evolving Connectionist Systems: Methods and Applications in Bioinformatics, Brain Study and Intelligent Machines , 2002, IEEE Transactions on Neural Networks.

[56]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[57]  E. Lughofer,et al.  Filtering of dynamic measurements in intelligent sensors for fault detection based on data-driven models , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[58]  Yevgeniy Bodyanskiy,et al.  THE CASCADE ORTHOGONAL NEURAL NETWORK , 2008 .

[59]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[60]  Bogdan Raducanu,et al.  Online pattern recognition and machine learning techniques for computer-vision: Theory and applications , 2010, Image Vis. Comput..

[61]  Nikola Kasabov,et al.  Evolving computational intelligence systems , 2005 .

[62]  Edwin Lughofer,et al.  Premise parameter estimation and adaptation in fuzzy systems with open-loop clustering methods , 2004, 2004 IEEE International Conference on Fuzzy Systems (IEEE Cat. No.04CH37542).

[63]  Nikola Kasabov,et al.  Evolving Connectionist Systems: The Knowledge Engineering Approach , 2007 .

[64]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[65]  Eyke Hüllermeier,et al.  Improving the interpretability of data-driven evolving fuzzy systems , 2005, EUSFLAT Conf..

[66]  Plamen P. Angelov,et al.  On-line identification of MIMO evolving Takagi- Sugeno fuzzy models , 2004, 2004 IEEE International Conference on Fuzzy Systems (IEEE Cat. No.04CH37542).

[67]  Plamen Angelov,et al.  Evolving Intelligent Systems: Methodology and Applications , 2010 .

[68]  Geoff Holmes,et al.  A Modified Quickprop Algorithm , 1991, Neural Computation.