Analysis of Generalized Pattern Searches

This paper contains a new convergence analysis for the Lewis and Torczon generalized pattern search (GPS) class of methods for unconstrained and linearly constrained optimization. This analysis is motivated by a desire to understand the successful behavior of the algorithm under hypotheses that are satisfied by many practical problems. Specifically, even if the objective function is discontinuous or extended-valued, the methods find a limit point with some minimizing properties. Simple examples show that the strength of the optimality conditions at a limit point depends not only on the algorithm, but also on the directions it uses and on the smoothness of the objective at the limit point in question. The contribution of this paper is to provide a simple convergence analysis that supplies detail about the relation of optimality conditions to objective smoothness properties and to the defining directions for the algorithm, and it gives previous results as corollaries.

[1]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[2]  John E. Dennis,et al.  Direct Search Methods on Parallel Machines , 1991, SIAM J. Optim..

[3]  P. Toint,et al.  A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds , 1991 .

[4]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[5]  C. T. Kelley,et al.  An Implicit Filtering Algorithm for Optimization of Functions with Many Local Minima , 1995, SIAM J. Optim..

[6]  V. Torczon,et al.  RANK ORDERING AND POSITIVE BASES IN PATTERN SEARCH ALGORITHMS , 1996 .

[7]  O. SIAMJ.,et al.  ON THE CONVERGENCE OF PATTERN SEARCH ALGORITHMS , 1997 .

[8]  Owen J. Eslinger,et al.  Optimization of Automotive Valve Train Components with Implicit Filtering , 2000 .

[9]  John E. Dennis,et al.  Managing surrogate objectives to optimize a helicopter rotor design - Further experiments , 1998 .

[10]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .

[11]  Robert Michael Lewis,et al.  Pattern Search Algorithms for Bound Constrained Minimization , 1999, SIAM J. Optim..

[12]  John E. Dennis,et al.  A framework for managing models in nonlinear optimization of computationally expensive functions , 1999 .

[13]  Chih-Jen Lin,et al.  Newton's Method for Large Bound-Constrained Optimization Problems , 1999, SIAM J. Optim..

[14]  C. T. Kelley,et al.  Superlinear Convergence and Implicit Filtering , 1999, SIAM J. Optim..

[15]  Robert Michael Lewis,et al.  Pattern Search Methods for Linearly Constrained Minimization , 1999, SIAM J. Optim..

[16]  Thomas de Quincey.  [C] , 2000, The Works of Thomas De Quincey, Vol. 1: Writings, 1799–1820.

[17]  Charles Audet,et al.  A surrogate-model-based method for constrained optimization , 2000 .

[18]  Tamara G. Kolda,et al.  Asynchronous Parallel Pattern Search for Nonlinear Optimization , 2001, SIAM J. Sci. Comput..

[19]  C. J. Price,et al.  On the Convergence of Grid-Based Methods for Unconstrained Optimization , 2000, SIAM J. Optim..

[20]  J. Dennis,et al.  Mixed Variable Optimization of the Number and Composition of Heat Intercepts in a Thermal Insulation System , 2001 .

[21]  Charles Audet,et al.  Pattern Search Algorithms for Mixed Variable Programming , 2000, SIAM J. Optim..

[22]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[23]  Robert Michael Lewis,et al.  A Globally Convergent Augmented Lagrangian Pattern Search Algorithm for Optimization with General Constraints and Simple Bounds , 2002, SIAM J. Optim..

[24]  Sven Leyffer,et al.  Nonlinear programming without a penalty function , 2002, Math. Program..

[25]  Sven Leyffer,et al.  On the Global Convergence of a Filter--SQP Algorithm , 2002, SIAM J. Optim..

[26]  V. Torczon,et al.  A GLOBALLY CONVERGENT AUGMENTED LAGRANGIAN ALGORITHM FOR OPTIMIZATION WITH GENERAL CONSTRAINTS AND SIMPLE BOUNDS , 2002 .

[27]  Nicholas I. M. Gould,et al.  Global Convergence of a Trust-Region SQP-Filter Algorithm for General Nonlinear Programming , 2002, SIAM J. Optim..

[28]  Charles Audet,et al.  Convergence Results for Pattern Search Algorithms are Tight , 2002 .

[29]  Roger Fletcher,et al.  On the global convergence of an SLP–filter algorithm that takes EQP steps , 2003, Math. Program..

[30]  Robert Michael Lewis,et al.  On the Local Convergence of Pattern Search , 2003, SIAM J. Optim..

[31]  C. J. Price,et al.  Frames and Grids in Unconstrained and Linearly Constrained Optimization: A Nonsmooth Approach , 2003, SIAM J. Optim..

[32]  Charles Audet,et al.  A Pattern Search Filter Method for Nonlinear Programming without Derivatives , 2001, SIAM J. Optim..