Nature-inspired multi-objective optimisation and transparent knowledge discovery via hierarchical fuzzy modelling

Knowledge discovery is one of the most important human activities, which helps people recognise and understand some of the intricacies associated with the ancient and modern worlds. With the rapid development in the human capabilities to both generate and collect data, the discovery of knowledge from data has become a practical and popular research topic. In this thesis, knowledge discovery from data is conducted from the following two overarching viewpoints: first, developing prediction models using the data that represent input-output relationships; second, based on these developed prediction models, finding the optimal designs (solutions) from a set of predefined objectives. The theoretical aspects behind the previous two research facets are described and the associated experimental studies are carried out. A particular focus of this thesis is on a cooperative fuzzy modelling framework, which integrates transparent (interpretable) fuzzy systems with robust evolutionary computing based algorithms involving several techniques, such as data clustering, data mining, and multi-objective optimisation. Evolutionary optimisation algorithms are also developed on the basis of nature and social inspired ideas. Optimisation forms an essential part of the modelling framework and is employed in the direct optimal design problems as well. The proposed cooperative fuzzy modelling methodology and the devised evolutionary optimisation algorithms are then applied to knowledge discovery in terms of systems modelling and control (static optimisation via reverse-engineering), using simulation platforms as well as real industrial data. The experimental results show that the proposed modelling framework and optimisation algorithms outperform some of the other salient techniques; the proposed approaches can successfully work within the context of the high-dimensional industrial applications, including modelling and optimal design problems.

[1]  Magne Setnes,et al.  Rule-based modeling: precision and transparency , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[2]  Tor Arne Johansen,et al.  Multiobjective identification of Takagi-Sugeno fuzzy models , 2003, IEEE Trans. Fuzzy Syst..

[3]  J. H. Ward.  Hierarchical Grouping to Optimize an Objective Function , 1963 .

[4]  Hisao Ishibuchi,et al.  Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining , 2004, Fuzzy Sets Syst..

[5]  John Yen,et al.  Application of statistical information criteria for optimal fuzzy model construction , 1998, IEEE Trans. Fuzzy Syst..

[6]  Mahdi Mahfouf,et al.  Mamdani-Type Fuzzy Modelling via Hierarchical Clustering and Multi-Objective Particle Swarm Optimisation (FM-HCPSO) , 2008 .

[7]  Michael N. Vrahatis,et al.  On the computation of all global minimizers through particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[8]  Chang-Hyun Kim,et al.  Evolving Compact and Interpretable Takagi–Sugeno Fuzzy Models With a New Encoding Scheme , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[9]  L. Wang,et al.  Fuzzy systems are universal approximators , 1992, [1992 Proceedings] IEEE International Conference on Fuzzy Systems.

[10]  David W. Corne,et al.  Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy , 2000, Evolutionary Computation.

[11]  Bernhard Sendhoff,et al.  On generating FC3 fuzzy rule systems from data using evolution strategies , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[12]  Gregg D. Wilensky,et al.  Neural Network Studies , 1993 .

[13]  Li-Xin Wang,et al.  A Course In Fuzzy Systems and Control , 1996 .

[14]  Manuel Valenzuela-Rendón.  The Fuzzy Classifier System: Motivations and first Results , 1990, PPSN.

[15]  Takanori Shibata,et al.  Genetic Algorithms And Fuzzy Logic Systems Soft Computing Perspectives , 1997 .

[16]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[17]  Y. Rahmat-Samii,et al.  Particle swarm optimization in electromagnetics , 2004, IEEE Transactions on Antennas and Propagation.

[18]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[19]  R. Eberhart,et al.  Fuzzy adaptive particle swarm optimization , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[20]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[21]  H. Ishibuchi,et al.  Multi-objective genetic algorithm and its applications to flowshop scheduling , 1996 .

[22]  James Kennedy,et al.  Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[23]  Euntai Kim,et al.  A new approach to fuzzy modeling , 1997, IEEE Trans. Fuzzy Syst..

[24]  Günter Rudolph,et al.  Contemporary Evolution Strategies , 1995, ECAL.

[25]  John Yen,et al.  Improving the interpretability of TSK fuzzy models by combining global learning and local learning , 1998, IEEE Trans. Fuzzy Syst..

[26]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[27]  F. B. Pickering,et al.  Physical metallurgy and the design of steels , 1978 .

[28]  Mahdi Mahfouf,et al.  A new Reduced Space Searching Algorithm (RSSA) and its application in optimal design of alloy steels , 2007, 2007 IEEE Congress on Evolutionary Computation.

[29]  Antonio F. Gómez-Skarmeta,et al.  Accurate, Transparent, and Compact Fuzzy Models for Function Approximation and Dynamic Modeling through Multi-objective Evolutionary Optimization , 2001, EMO.

[30]  Günter Rudolph,et al.  Convergence properties of some multi-objective evolutionary algorithms , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[31]  Juan R. Velasco.  Genetic-based on-line learning for fuzzy process control , 1998, Int. J. Intell. Syst..

[32]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[33]  Lothar Thiele,et al.  An evolutionary algorithm for multiobjective optimization: the strength Pareto approach , 1998 .

[34]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1992, Artificial Intelligence.

[35]  R. Storn,et al.  On the usage of differential evolution for function optimization , 1996, Proceedings of North American Fuzzy Information Processing.

[36]  Fionn Murtagh,et al.  A Survey of Recent Advances in Hierarchical Clustering Algorithms , 1983, Comput. J..

[37]  T. Ray.  Constrained robust optimal design using a multiobjective evolutionary algorithm , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[38]  George Nagy,et al.  State of the art in pattern recognition , 1968 .

[39]  R. Storn,et al.  Differential Evolution - A simple and efficient adaptive scheme for global optimization over continuous spaces , 2004 .

[40]  Mahdi Mahfouf,et al.  FUZZY PREDICTIVE MODELLING USING HIERARCHICAL CLUSTERING AND MULTI-OBJECTIVE OPTIMISATION FOR MECHANICAL PROPERTIES OF ALLOY STEELS , 2007 .

[41]  Mark Richards,et al.  Choosing a starting configuration for particle swarm optimization , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[42]  Kalyan Veeramachaneni,et al.  Fitness-distance-ratio based particle swarm optimization , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[43]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[44]  Michio Sugeno,et al.  A fuzzy-logic-based approach to qualitative modeling , 1993, IEEE Trans. Fuzzy Syst..

[45]  Ralf Mikut,et al.  Interpretability issues in data-based learning of fuzzy systems , 2005, Fuzzy Sets Syst..

[46]  Jonathan Tenner,et al.  Optimisation of the heat treatment of steel using neural networks , 2000 .

[47]  Charles L. Karr,et al.  Genetic algorithms for fuzzy controllers , 1991 .

[48]  Hisao Ishibuchi,et al.  Fuzzy data mining: effect of fuzzy discretization , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[49]  Uzay Kaymak,et al.  Similarity measures in fuzzy rule base simplification , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[50]  Kim-Fung Man,et al.  Multi-objective hierarchical genetic algorithm for interpretable fuzzy rule-based knowledge extraction , 2005, Fuzzy Sets Syst..

[51]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[52]  John Yen,et al.  Simplifying fuzzy rule-based models using orthogonal transformation methods , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[53]  Shangxu Hu,et al.  A New Approach to Improve Particle Swarm Optimization , 2003, GECCO.

[54]  Jürgen Teich,et al.  Strategies for finding good local guides in multi-objective particle swarm optimization (MOPSO) , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[55]  Wenjun Zhang,et al.  Dissipative particle swarm optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[56]  Magne Setnes,et al.  GA-fuzzy modeling and classification: complexity and performance , 2000, IEEE Trans. Fuzzy Syst..

[57]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[58]  Ioannis B. Theocharis,et al.  A GA-based fuzzy modeling approach for generating TSK models , 2002, Fuzzy Sets Syst..

[59]  Gilles Venturini,et al.  SIA: A Supervised Inductive Algorithm with Genetic Search for Learning Attributes based Concepts , 1993, ECML.

[60]  Kim-Fung Man,et al.  Agent-based evolutionary approach for interpretable rule-based knowledge extraction , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[61]  Thomas Kiel Rasmussen,et al.  Hybrid Particle Swarm Optimiser with breeding and subpopulations , 2001 .

[62]  Li-Xin Wang,et al.  Adaptive fuzzy systems and control , 1994 .

[63]  Ebrahim Mamdani,et al.  Applications of fuzzy algorithms for control of a simple dynamic plant , 1974 .

[64]  G. Langholz,et al.  Genetic-Based New Fuzzy Reasoning Models with Application to Fuzzy Control , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[65]  Philip R. Thrift,et al.  Fuzzy Logic Synthesis with Genetic Algorithms , 1991, ICGA.

[66]  John Yen,et al.  Extracting fuzzy rules for system modeling using a hybrid of genetic algorithms and Kalman filter , 1999, Fuzzy Sets Syst..

[67]  Stefano Marsili-Libelli,et al.  Adaptive fuzzy pattern recognition in the anaerobic digestion process , 1996, Pattern Recognit. Lett..

[68]  Ebrahim H. Mamdani,et al.  An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller , 1999, Int. J. Hum. Comput. Stud..

[69]  Benjamin King.  Step-Wise Clustering Procedures , 1967 .

[70]  Hisao Ishibuchi,et al.  Interpretability Issues in Fuzzy Genetics-Based Machine Learning for Linguistic Modelling , 2003, Modelling with Words.

[71]  K. C. Ng,et al.  Design of sophisticated fuzzy logic controllers using genetic algorithms , 1994, Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference.

[72]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[73]  Mahdi Mahfouf,et al.  ADAPTIVE WEIGHTED PARTICLE SWARM MULTIOBJECTIVE OPTIMISATION AND SOCIETAL REASONING FOR THE DESIGN OF ALLOY STEELS , 2006 .

[74]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[75]  M. Mahfouf,et al.  A New Structure for Particle Swarm Optimization (nPSO) Applicable to Single Objective and Multiobjective Problems , 2006, 2006 3rd International IEEE Conference Intelligent Systems.

[76]  V. Novák,et al.  Mathematical Principles of Fuzzy Logic , 1999 .

[77]  Maysam F. Abbod,et al.  OPTIMISATION OF STEEL PRODUCTION INCORPORATING ECONOMIC FACTORS , 2002 .

[78]  Héctor Pomares,et al.  Self-organized fuzzy system generation from training examples , 2000, IEEE Trans. Fuzzy Syst..

[79]  L X Wang,et al.  Fuzzy basis functions, universal approximation, and orthogonal least-squares learning , 1992, IEEE Trans. Neural Networks.

[80]  Suganthan.  [IEEE 1999. Congress on Evolutionary Computation-CEC99 - Washington, DC, USA (6-9 July 1999)] Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406) - Particle swarm optimiser with neighbourhood operator , 1999 .

[81]  Marco Laumanns,et al.  A Tutorial on Evolutionary Multiobjective Optimization , 2004, Metaheuristics for Multiobjective Optimisation.

[82]  J. Kennedy,et al.  Population structure and particle swarm performance , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[83]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[84]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[85]  Lotfi A. Zadeh,et al.  Outline of a New Approach to the Analysis of Complex Systems and Decision Processes , 1973, IEEE Trans. Syst. Man Cybern..

[86]  Stephen F. Smith,et al.  A learning system based on genetic adaptive algorithms , 1980 .

[87]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..