Automatic construction of feedforward/recurrent fuzzy systems by clustering-aided simplex particle swarm optimization

This paper proposes a new approach for automating the structure and parameter learning of fuzzy systems by clustering-aided simplex particle swarm optimization, called CSPSO. Unlike most evolutionary fuzzy systems, where the structure of the fuzzy system is assigned in advance, an on-line fuzzy clustering approach is proposed for system structure learning. This structure learning not only helps determine the number of rules automatically, but also avoids the generation of highly similar fuzzy sets on each input variable. In addition, it improves subsequent parameter learning performance by assigning suitable initial locations of the fuzzy sets on each input variable. Once a new rule is generated, the corresponding parameters are further tuned by the hybrid of the simplex method and particle swarm optimization (PSO). In CSPSO, each fuzzy system corresponds to a particle in PSO, and the idea of the simplex method is incorporated to improve PSO searching performance. To verify the performance of CSPSO, two simulations on feedforward fuzzy systems design are performed. In addition, design of a recurrent fuzzy controller for a practical experiment on water bath temperature control is performed. Comparisons with other design approaches are also made in these examples.

[1]  Yang Xu,et al.  Application of fuzzy Naive Bayes and a real-valued genetic algorithm in identification of fuzzy model , 2005, Inf. Sci..

[2]  Shu-Kai S. Fan,et al.  Hybrid simplex search and particle swarm optimization for the global optimization of multimodal functions , 2004 .

[3]  Paulo Cortez,et al.  Particle swarms for feedforward neural network training , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[4]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[5]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[6]  H. Ishibuchi Genetic fuzzy systems: evolutionary tuning and learning of fuzzy knowledge bases , 2004 .

[7]  Chulhyun Kim,et al.  Forecasting time series with genetic fuzzy predictor ensemble , 1997, IEEE Trans. Fuzzy Syst..

[8]  F. Klawonn,et al.  Fuzzy Cluster Analysis: Methods for Classification, Data Analysis and Image Recognition , 1999 .

[9]  Ahmad Lotfi,et al.  Learning fuzzy inference systems using an adaptive membership function scheme , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[10]  Chia-Feng Juang,et al.  A TSK-type recurrent fuzzy network for dynamic systems processing by neural network and genetic algorithms , 2002, IEEE Trans. Fuzzy Syst..

[11]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[12]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[13]  Konstantinos E. Parsopoulos,et al.  Initializing the Particle Swarm Optimizer Using the Nonlinear Simplex Method , 2002 .

[14]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[15]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[16]  R. Tibshirani,et al.  Generalized Additive Models , 1991 .

[17]  Andries Petrus Engelbrecht,et al.  A study of particle swarm optimization particle trajectories , 2006, Inf. Sci..

[18]  Marco Russo,et al.  Genetic fuzzy learning , 2000, IEEE Trans. Evol. Comput..

[19]  Plamen P. Angelov,et al.  Automatic generation of fuzzy rule-based models from data by genetic algorithms , 2003, Inf. Sci..

[20]  Chin-Teng Lin,et al.  An On-Line Self-Constructing Neural Fuzzy Inference Network and Its Applications , 1998 .

[21]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[22]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[23]  G. R. Hext,et al.  Sequential Application of Simplex Designs in Optimisation and Evolutionary Operation , 1962 .

[24]  Jacek M. Zurada,et al.  Computational Intelligence: Imitating Life , 1994 .

[25]  Chin-Teng Lin,et al.  Genetic Reinforcement Learning through Symbiotic Evolution for Fuzzy Controller Design , 2022 .

[26]  Tapabrata Ray,et al.  A Swarm Metaphor for Multiobjective Design Optimization , 2002 .

[27]  W. Pinebrook The evolution of strategy. , 1990, Case studies in health administration.

[28]  Chin-Teng Lin,et al.  An online self-constructing neural fuzzy inference network and its applications , 1998, IEEE Trans. Fuzzy Syst..

[29]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[30]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[31]  D. Fogel Evolutionary algorithms in theory and practice , 1997, Complex..

[32]  Young-Il Kim,et al.  A cluster validation index for GK cluster analysis based on relative degree of sharing , 2004, Inf. Sci..

[33]  María José del Jesús,et al.  Genetic tuning of fuzzy rule deep structures preserving interpretability and its interaction with fuzzy rule set reduction , 2005, IEEE Transactions on Fuzzy Systems.

[34]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[35]  Abdollah Homaifar,et al.  Simultaneous design of membership functions and rule sets for fuzzy controllers using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[36]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.