Hybrid optimization schemes for simulation-based problems

The inclusion of computer simulations in the study and design of complex engineering systems has created a need for efficient approaches to simulation-based optimization. For example, in water resources management problems, optimization problems regularly consist of objective functions and constraints that rely on output from a PDE-based simulator. Various assumptions can be made to simplify either the objective function or the physical system so that gradient-based methods apply, however the incorporation of realistic objection functions can be accomplished given the availability of derivative-free optimization methods. A wide variety of derivative-free methods exist and each method has both advantages and disadvantages. Therefore, to address such problems, we propose a hybrid approach, which allows the combining of beneficial elements of multiple methods in order to more efficiently search the design space. Specifically, in this paper, we illustrate the capabilities of two novel algorithms; one which hybridizes pattern search optimization with Gaussian Process emulation and the other which hybridizes pattern search and a genetic algorithm. We describe the hybrid methods and give some numerical results for a hydrological application which illustrate that the hybrids find an optimal solution under conditions for which traditional optimal search methods fail.

[1]  Oskar von Stryk,et al.  A mixed-integer simulation-based optimization approach with surrogate functions in water resources management , 2008 .

[2]  Sharif Rahman,et al.  A hybrid surrogate and pattern search optimization method and application to microelectronics , 2006 .

[3]  C. T. Kelley,et al.  An Implicit Filtering Algorithm for Optimization of Functions with Many Local Minima , 1995, SIAM J. Optim..

[4]  John J. Grefenstette,et al.  Optimization of Control Parameters for Genetic Algorithms , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[5]  Thomas Hemker,et al.  Framework for Particle Swarm Optimization with Surrogate Functions , 2009 .

[6]  Kenneth A. De Jong,et al.  An Analysis of the Interacting Roles of Population Size and Crossover in Genetic Algorithms , 1990, PPSN.

[7]  K. R. Fowler,et al.  Approaching the Groundwater Remediation Problem Using Multifidelity Optimization. , 2006 .

[8]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[9]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[10]  Tamara G. Kolda,et al.  Revisiting Asynchronous Parallel Pattern Search for Nonlinear Optimization , 2005, SIAM J. Optim..

[11]  O. SIAMJ.,et al.  ON THE CONVERGENCE OF PATTERN SEARCH ALGORITHMS , 1997 .

[12]  T. Crainic,et al.  Parallel Meta-Heuristics , 2010 .

[13]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[14]  O. V. Stryk,et al.  DERIVATIVE-FREE OPTIMIZATION METHODS FOR HANDLING FIXED COSTS IN OPTIMAL GROUNDWATER REMEDIATION DESIGN , 2006 .

[15]  Mike Rees,et al.  5. Statistics for Spatial Data , 1993 .

[16]  Paul P. Wang,et al.  MT3DMS: A Modular Three-Dimensional Multispecies Transport Model for Simulation of Advection, Dispersion, and Chemical Reactions of Contaminants in Groundwater Systems; Documentation and User's Guide , 1999 .

[17]  Robert Haining,et al.  Statistics for spatial data: by Noel Cressie, 1991, John Wiley & Sons, New York, 900 p., ISBN 0-471-84336-9, US $89.95 , 1993 .

[18]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[19]  Cass T. Miller,et al.  Optimal design for problems involving flow and transport phenomena in saturated subsurface systems , 2002 .

[20]  Tamara G. Kolda,et al.  Algorithm 856: APPSPACK 4.0: asynchronous parallel pattern search for derivative-free optimization , 2006, TOMS.

[21]  Tamara G. Kolda,et al.  Asynchronous Parallel Pattern Search for Nonlinear Optimization , 2001, SIAM J. Sci. Comput..

[22]  Mary C. Hill,et al.  MODFLOW-2000, the U.S. Geological Survey modular ground-water model : user guide to the LMT6 package, the linkage with MT3DMS for multi-species mass transport modeling , 2001 .

[23]  Margaret J. Eppstein,et al.  A hybrid genetic algorithm with pattern search for finding heavy atoms in protein crystals , 2005, GECCO '05.

[24]  John E. Dennis,et al.  Direct Search Methods on Parallel Machines , 1991, SIAM J. Optim..

[25]  James R. Craig,et al.  Pump-and-treat optimization using analytic element method flow models , 2006 .

[26]  Owen J. Eslinger,et al.  IFFCO: implicit filtering for constrained optimization, version 2 , 1999 .

[27]  Robert B. Gramacy,et al.  Bayesian treed gaussian process models , 2005 .

[28]  Robert B. Gramacy,et al.  Ja n 20 08 Bayesian Treed Gaussian Process Models with an Application to Computer Modeling , 2009 .

[29]  Griffin Caprio,et al.  Parallel Metaheuristics , 2008, IEEE Distributed Systems Online.

[30]  David P. Ahlfeld,et al.  PRESENCE OF NONCONVEXITY IN GROUNDWATER CONCENTRATION RESPONSE FUNCTIONS , 1998 .

[31]  Cass T. Miller,et al.  Solution of a Groundwater Control Problem with Implicit Filtering , 2002 .

[32]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[33]  Tamara G. Kolda,et al.  Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods , 2003, SIAM Rev..

[34]  William J. Welch,et al.  Computer experiments and global optimization , 1997 .

[35]  P. Gilmore,et al.  IFFCO (Implicit filtering for constrained optimization), users' guide , 1993 .

[36]  Charles Audet,et al.  Comparison of derivative-free optimization methods for groundwater supply and hydraulic capture community problems , 2008 .

[37]  Edward I. George,et al.  Bayesian Treed Models , 2002, Machine Learning.

[38]  Virginia Torczon,et al.  PDS: Direct Search Methods for Unconstrained Optimization on Either Sequential or Parallel Machines , 1992 .

[39]  Carl Tim Kelley,et al.  Iterative methods for optimization , 1999, Frontiers in applied mathematics.

[40]  D. Ahlfeld,et al.  Advective control of groundwater contaminant plumes: Model development and comparison to hydraulic control , 1999 .