DYNAMIC MODELLING USING GENETIC PROGRAMMING

Abstract In this contribution we demonstrate how a Single Objective Genetic Programming (SOGP) and a Multi-Objective Genetic Programming (MOGP) algorithm can be used to evolve accurate input-output models of dynamic processes. Having described the algorithms, two case studies are used to compare their performance with that of Filter-Based Neural Networks (FBNNs). For the examples given, the models generated using GP have comparable prediction performance to the FBNN. However, performance with respect to additional modelling criteria can be improved using the MOGP algorithm.

[1]  Carlos M. Fonseca,et al.  Multiobjective genetic algorithms , 1993 .

[2]  Sanjeev Garg,et al.  Multiobjective optimization of a free radical bulk polymerization reactor using genetic algorithm , 1999 .

[3]  Peter J. Fleming,et al.  System identification strategies applied to aircraft gas turbine engines , 2000 .

[4]  David E. Goldberg,et al.  A niched Pareto genetic algorithm for multiobjective optimization , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[5]  Tommy W. S. Chow,et al.  A weight initialization method for improving training speed in feedforward neural network , 2000, Neurocomputing.

[6]  Ken Sharman,et al.  Evolving signal processing algorithms by genetic programming , 1995 .

[7]  Larry J. Eshelman,et al.  Using genetic search to exploit the emergent behavior of neural networks , 1990 .

[8]  Kalyanmoy Deb,et al.  Multi-objective Genetic Algorithms: Problem Difficulties and Construction of Test Problems , 1999, Evolutionary Computation.

[9]  A. J. Morris,et al.  Artificial neural networks in process engineering , 1991 .

[10]  Lee Altenberg,et al.  Evolving better representations through selective genome growth , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[11]  Venkat Venkatasubramanian,et al.  A genetic algorithmic framework for process design and optimization , 1991 .

[12]  S. A. Billings,et al.  Multi-directional model validity tests for non-linear system identification , 2000 .

[13]  Peter J. Fleming,et al.  Assessing the performance of multiobjective genetic algorithms for optimization of a batch process scheduling problem , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[14]  Peter J. Fleming,et al.  Multiobjective optimization and multiple constraint handling with evolutionary algorithms. I. A unified formulation , 1998, IEEE Trans. Syst. Man Cybern. Part A.

[15]  Peter J. Fleming,et al.  Multiobjective genetic programming for gas turbine engine model identification , 1998 .

[16]  N. Draper,et al.  Applied Regression Analysis , 1966 .

[17]  Gary G. R. Green,et al.  How Neural Networks Work : The Mathematics of Networks Used to Solve Standard Engineering Problems , 1991 .

[18]  Stephen A. Billings,et al.  Nonlinear model validation using correlation tests , 1994 .

[19]  Anders Krogh,et al.  A Simple Weight Decay Can Improve Generalization , 1991, NIPS.

[20]  Peter J. Angeline,et al.  Explicitly Defined Introns and Destructive Crossover in Genetic Programming , 1996 .

[21]  Kalyanmoy Deb,et al.  Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms , 1994, Evolutionary Computation.

[22]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[23]  John R. Koza,et al.  Genetic Programming III: Darwinian Invention & Problem Solving , 1999 .

[24]  M. A. Kaboudan,et al.  A Measure of Time Series’ Predictability Using Genetic Programming , 2004 .

[25]  Peter Nordin,et al.  Complexity Compression and Evolution , 1995, ICGA.

[26]  K S Narendra,et al.  IDENTIFICATION AND CONTROL OF DYNAMIC SYSTEMS USING NEURAL NETWORKS , 1990 .

[27]  P. Nordin,et al.  Explicitly defined introns and destructive crossover in genetic programming , 1996 .

[28]  Ali A. Minai,et al.  Back-propagation heuristics: a study of the extended delta-bar-delta algorithm , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[29]  C. Charalambous,et al.  Conjugate gradient algorithm for efficient training of artifi-cial neural networks , 1990 .

[30]  Stephen A. Billings,et al.  Non-linear system identification using neural networks , 1990 .

[31]  Won-Pil Baek,et al.  Critical heat flux prediction using genetic programming for water flow in vertical round tubes , 1997 .

[32]  Julian Morris,et al.  Dynamic neural networks in non-linear predictive control (an industrial application) , 1996 .

[33]  C. Fonseca,et al.  Non-Linear System Identification with Multiobjective Genetic Algorithms , 1996 .

[34]  David B. Fogel,et al.  Evolutionary Computation: Towards a New Philosophy of Machine Intelligence , 1995 .

[35]  John R. Koza,et al.  Automatic Synthesis, Placement, and Routing of an Amplifier Circuit by Means of Genetic Programming , 2000, ICES.

[36]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[37]  C. Hwang.  Multiple Objective Decision Making - Methods and Applications: A State-of-the-Art Survey , 1979 .

[38]  M. Willis,et al.  Systems modelling using genetic programming , 1997 .

[39]  Lothar Thiele,et al.  A Comparison of Selection Schemes used in Genetic Algorithms , 1995 .

[40]  Barry Lennox,et al.  Extruder modelling: a comparison of two paradigms , 1996 .

[41]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[42]  Gary B. Lamont,et al.  Multiobjective Evolutionary Algorithms: Analyzing the State-of-the-Art , 2000, Evolutionary Computation.

[43]  N. V. Bhat,et al.  Use of neural nets for dynamic modeling and control of chemical process systems , 1990 .

[44]  Rolf Johansson,et al.  System modeling and identification , 1993 .

[45]  David E. Goldberg,et al.  Real-coded Genetic Algorithms, Virtual Alphabets, and Blocking , 1991, Complex Syst..

[46]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[47]  Ian C. Parmee,et al.  IDENTIFICATION OF FLUID SYSTEMS USING GENETIC PROGRAMMING , 1996 .

[48]  Nichael Lynn Cramer,et al.  A Representation for the Adaptive Generation of Simple Sequential Programs , 1985, ICGA.

[49]  S. Billings,et al.  A prediction-error and stepwise-regression estimation algorithm for non-linear systems , 1986 .

[50]  Chris Aldrich,et al.  Empirical modelling of chemical process systems with evolutionary programming , 1998 .

[51]  S. Billings,et al.  Algorithms for minimal model structure detection in nonlinear dynamic system identification , 1997 .

[52]  S. Billings,et al.  Correlation based model validity tests for non-linear models , 1986 .

[53]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[54]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[55]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[56]  Tobias Blickle,et al.  Evolving Compact Solutions in Genetic Programming: A Case Study , 1996, PPSN.

[57]  P. Hajela,et al.  Genetic search strategies in multicriterion optimal design , 1991 .

[58]  Lishan Kang,et al.  The Kinetic Evolutionary Modeling of Complex Systems of Chemical Reactions , 1999, Comput. Chem..

[59]  Bhaskar D. Kulkarni,et al.  Consider genetic programming for process identification , 1999 .

[60]  P. P. Kanjilal,et al.  Adaptive Prediction and Predictive Control , 1995 .

[61]  John R. Koza,et al.  Genetic programming 2 - automatic discovery of reusable programs , 1994, Complex Adaptive Systems.

[62]  Paul M. Frank,et al.  New developments using AI in fault diagnosis , 1996 .

[63]  Dale E. Seborg,et al.  Nonlinear Process Control , 1996 .

[64]  Terence Soule,et al.  Code growth in genetic programming , 1996 .

[65]  Peter J. Fleming,et al.  Multiobjective genetic algorithms made easy: selection sharing and mating restriction , 1995 .

[66]  G. Raidl.  A Hybrid GP Approach for Numerically Robust Symbolic Regression , 2002 .

[67]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[68]  David E. Rumelhart,et al.  Generalization by Weight-Elimination with Application to Forecasting , 1990, NIPS.

[69]  Hisao Ishibuchi,et al.  Multi-objective genetic local search algorithm , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[70]  C. Hwang,et al.  Fuzzy Multiple Objective Decision Making: Methods And Applications , 1996 .

[71]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[72]  David J. Murray-Smith,et al.  Nonlinear model structure identification using genetic programming , 1998 .

[73]  David J. Montana,et al.  Evolving control laws for a network of traffic signals , 1996 .

[74]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[75]  Kay Chen Tan,et al.  System identification and linearisation using genetic algorithms with simulated annealing , 1995 .

[76]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[77]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[78]  陳樹衡,et al.  Using Genetic Programming to Model Volatility in Financial Time Series , 1997 .

[79]  Peter J. Fleming,et al.  An Overview of Evolutionary Algorithms in Multiobjective Optimization , 1995, Evolutionary Computation.

[80]  Marco Laumanns,et al.  A unified model for multi-objective evolutionary algorithms with elitism , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[81]  Kurt Dirk Bettenhausen,et al.  Self-organizing structured modelling of a biotechnological fed-batch fermentation by means of genetic programming , 1995 .

[82]  Petre Stoica,et al.  Decentralized Control , 2018, The Control Systems Handbook.

[83]  Mark J. Willis,et al.  Steady-state modelling of chemical process systems using genetic programming , 1997 .

[84]  Bernard Widrow,et al.  Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[85]  Jean-Michel Renders,et al.  Genetic Algorithms and Their Potential for Use in Process Control: A Case Study , 1991, ICGA.

[86]  Julian Morris,et al.  Artificial neural networks in process estimation and control , 1992, Autom..

[87]  K. Smillie,et al.  An introduction to regression and correlation , 1966 .

[88]  Peter J. Fleming,et al.  Gas turbine engine controller design using multiobjective genetic algorithms , 1995 .

[89]  Sheng Chen,et al.  Representations of non-linear systems: the NARMAX model , 1989 .

[90]  Sheng Chen,et al.  Practical identification of NARMAX models using radial basis functions , 1990 .

[91]  Rafiqul Gani,et al.  A generalized model for distillation columns—I: Model description and applications , 1986 .

[92]  Dominic P. Searson,et al.  Nonlinear continuum regression: an evolutionary approach , 2000 .

[93]  Mark J. Willis,et al.  Modeling and control of a food extrusion process , 1997 .

[94]  James E. Baker,et al.  Reducing Bias and Inefficienry in the Selection Algorithm , 1987, ICGA.

[95]  Thomas Bäck,et al.  Evolutionary computation: Toward a new philosophy of machine intelligence , 1997, Complex..

[96]  Hitoshi Iba,et al.  System identification approach to genetic programming , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[97]  Dominic P. Searson,et al.  Chemical process controller design using genetic programming , 1998 .

[98]  Sebastian Engell,et al.  Sequencing of batch operations for a highly coupled production process: Genetic algorithms versus mathematical programming , 1998 .

[99]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[100]  J. B. Gomm,et al.  Experiment design considerations for non-linear system identification using neural networks , 1997 .

[101]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[102]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[103]  Zbigniew Michalewicz,et al.  Evolutionary Computation 2 , 2000 .

[104]  Yaman Arkun,et al.  Control of nonlinear systems using polynomial ARMA models , 1993 .

[105]  J. Gibbons,et al.  Nonparametric Methods for Quantitative Analysis. , 1997 .

[106]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[107]  G. Bierman.  Factorization methods for discrete sequential estimation , 1977 .

[108]  Julian Morris,et al.  Nonlinear and direction-dependent dynamic process modelling using neural networks , 1996 .

[109]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[110]  Riccardo Poli,et al.  Exact Schema Theorem and Effective Fitness for GP with One-Point Crossover , 2000, GECCO.