Automatic model type selection with heterogeneous evolution: An application to RF circuit block modeling

Many complex, real world phenomena are difficult to study directly using controlled experiments. Instead, the use of computer simulations has become commonplace as a cost effective alternative. However, regardless of Moorepsilas law, performing high fidelity simulations still requires a great investment of time and money. Surrogate modeling (metamodeling) has become indispensable as an alternative solution for relieving this burden. Many surrogate model types exist (support vector machines, Kriging, RBF models, neural networks, ...) but no type is optimal in all circumstances. Nor is there any hard theory available that can help make this choice. The same is true for setting the surrogate model parameters (bias- variance trade-off). Traditionally, the solution to both problems has been a pragmatic one, guided by intuition, prior experience or simply available software packages. In this paper we present a more founded approach to these problems. We describe an adaptive surrogate modeling environment, driven by speciated evolution, to automatically determine the optimal model type and complexity. Its utility and performance is presented on a case study from electronics.

[1]  G. Gary Wang,et al.  Review of Metamodeling Techniques in Support of Engineering Design Optimization , 2007, DAC 2006.

[2]  Russell R. Barton,et al.  A review on design, modeling and applications of computer experiments , 2006 .

[3]  Yung-Keun Kwon,et al.  Critical heat flux function approximation using genetic algorithms , 2005, IEEE Transactions on Nuclear Science.

[4]  Arthur C. Sanderson,et al.  Planning multiple paths with evolutionary speciation , 2001, IEEE Trans. Evol. Comput..

[5]  R. Shah,et al.  Least Squares Support Vector Machines , 2022 .

[6]  Reiko Tanese,et al.  Distributed Genetic Algorithms , 1989, ICGA.

[7]  Loren Paul Rees,et al.  A sequential-design metamodeling strategy for simulation optimization , 2004, Comput. Oper. Res..

[8]  Zdenek Konfrst,et al.  Parallel Genetic Algorithms: Advances, Computing Trends, Applications and Perspectives , 2004, IPDPS.

[9]  Paul K. Davis,et al.  Preface , 2021 .

[10]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[11]  Carolyn Conner Seepersad,et al.  Building Surrogate Models Based on Detailed and Approximate Simulations , 2004, DAC 2004.

[12]  Jürgen Branke,et al.  Efficient search for robust solutions by means of evolutionary algorithms and fitness approximation , 2006, IEEE Transactions on Evolutionary Computation.

[13]  Russell R. Barton.  Design of experiments for fitting subsystem metamodels , 1997, WSC '97.

[14]  Marios K. Karakasis,et al.  EVOLUTIONARY ALGORITHMS WITH SURROGATE MODELING FOR COMPUTATIONALLY EXPENSIVE OPTIMIZATION PROBLEMS , 2006 .

[15]  Enrique Alba,et al.  Parallelism and evolutionary algorithms , 2002, IEEE Trans. Evol. Comput..

[16]  Satoshi Tomioka,et al.  Nonlinear Least Square Regression by Adaptive Domain Method With Multiple Genetic Algorithms , 2007, IEEE Transactions on Evolutionary Computation.

[17]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[18]  Søren Nymand Lophaven,et al.  DACE - A Matlab Kriging Toolbox , 2002 .

[19]  Bernhard Sendhoff,et al.  A framework for evolutionary optimization with approximate fitness functions , 2002, IEEE Trans. Evol. Comput..

[20]  Matthew J. Streeter,et al.  Automated Discovery of Numerical Approximation Formulae via Genetic Programming , 2001, Genetic Programming and Evolvable Machines.

[21]  Giandomenico Spezzano,et al.  A scalable cellular implementation of parallel genetic programming , 2003, IEEE Trans. Evol. Comput..

[22]  T. Dhaene,et al.  Rational modeling of spectral data using orthonormal vector fitting , 2005, Proceedings. 9th IEEE Workshop on Signal Propagation on Interconnects, 2005..

[23]  Ranga Vemuri,et al.  An active learning scheme using support vector machines for analog circuit feasibility classification , 2005, 18th International Conference on VLSI Design held jointly with 4th International Conference on Embedded Systems Design.

[24]  Niels Kjølstad Poulsen,et al.  NNSYSID-Toolbox for System Identification with Neural Networks , 2002 .

[25]  R. Achar,et al.  Stability, Causality, and Passivity in Electrical Interconnect Models , 2007, IEEE Transactions on Advanced Packaging.

[26]  T. Simpson,et al.  Comparative studies of metamodelling techniques under multiple modelling criteria , 2001 .

[27]  Ashu Jain,et al.  A comparative analysis of training methods for artificial neural network rainfall-runoff models , 2006, Appl. Soft Comput..

[28]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[29]  Raphael T. Haftka,et al.  Surrogate-based Analysis and Optimization , 2005 .

[30]  Dirk Gorissen,et al.  Adaptive Distributed Metamodeling , 2006, VECPAR.

[31]  Kai-Yew Lum,et al.  Max-min surrogate-assisted evolutionary algorithm for robust design , 2006, IEEE Transactions on Evolutionary Computation.

[32]  Nowostawski,et al.  [IEEE 1999 Third International Conference on Knowledge-Based Intelligent Information Engineering Systems. KES\'99 - Adelaide, SA, Australia (31 Aug.-1 Sept. 1999)] 1999 Third International Conference on Knowledge-Based Intelligent Information Engineering Systems. Proceedings (Cat. No.99TH8410) - Par , 1999 .

[33]  S. Schwartz,et al.  An accelerated sequential algorithm for producing D -optimal designs , 1989 .

[34]  Robert B. Gramacy,et al.  Parameter space exploration with Gaussian process trees , 2004, ICML.

[35]  Riccardo Poli,et al.  Parallel genetic algorithm taxonomy , 1999, 1999 Third International Conference on Knowledge-Based Intelligent Information Engineering Systems. Proceedings (Cat. No.99TH8410).

[36]  Dirk Gorissen,et al.  HETEROGENEOUS EVOLUTION OF SURROGATE MODELS , 2007 .

[37]  Thomas H. Lee,et al.  The Design of CMOS Radio-Frequency Integrated Circuits: RF CIRCUITS THROUGH THE AGES , 2003 .

[38]  Timothy W. Simpson,et al.  Metamodels for Computer-based Engineering Design: Survey and recommendations , 2001, Engineering with Computers.

[39]  Dimitri N. Mavris,et al.  Aerospace Vehicle Concept Selection Using Parallel, Variable Fidelity Genetic Algorithms , 2004 .

[40]  Yun-Seog Yeun,et al.  Implementing linear models in genetic programming , 2004, IEEE Trans. Evol. Comput..

[41]  L. Darrell Whitley,et al.  Cellular genetic algorithms as function optimizers: locality effects , 1994, SAC '94.

[42]  Jack Dongarra,et al.  High Performance Computing for Computational Science , 2003 .

[43]  Brian Ellis.  The Design of CMOS Radio-Frequency Integrated Circuits , 2004 .

[44]  Shahab D. Mohaghegh,et al.  Development of Surrogate Reservoir Models (SRM) For Fast Track Analysis of Complex Reservoirs , 2006 .

[45]  Sven F. Crone,et al.  Genetic Algorithms for Support Vector Machine Model Selection , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.