A derivative-free algorithm for spherically constrained optimization

Spherically constrained optimization, which minimizes an objective function on a unit sphere, has wide applications in numerical multilinear algebra, signal processing, solid mechanics, etc. In this paper, we consider a certain case that the derivatives of the objective function are unavailable. This case arises frequently in computational science, chemistry, physics, and other enormous areas. To explore the spherical structure of the above problem, we apply the Cayley transform to preserve iterates on the sphere and propose a derivative-free algorithm, which employs a simple model-based trust-region framework. Under mild conditions, global convergence of the proposed algorithm is established. Preliminary numerical experiments illustrate the promising performances of our algorithm.

[1]  Andreas Griewank,et al.  Automatic Differentiation of Algorithms: From Simulation to Optimization , 2000, Springer New York.

[2]  Paul Tseng,et al.  Gilding the Lily: A Variant of the Nelder-Mead Algorithm Based on Golden-Section Search , 2002, Comput. Optim. Appl..

[3]  Xiaojin Huang,et al.  An interior affine scaling cubic regularization algorithm for derivative-free optimization subject to bound constraints , 2017, J. Comput. Appl. Math..

[4]  M. J. D. Powell,et al.  UOBYQA: unconstrained optimization by quadratic approximation , 2002, Math. Program..

[5]  Kazuo Yamaguchi,et al.  Borda winner in facility location problems on sphere , 2016, Soc. Choice Welf..

[6]  Xiaoyan Zhang,et al.  The Fiedler Vector of a Laplacian Tensor for Hypergraph Partitioning , 2017, SIAM J. Sci. Comput..

[7]  Jorge Nocedal,et al.  Wedge trust region methods for derivative free optimization , 2002, Math. Program..

[8]  Stefan M. Wild,et al.  Benchmarking Derivative-Free Optimization Algorithms , 2009, SIAM J. Optim..

[9]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[10]  Katya Scheinberg,et al.  A derivative free optimization algorithm in practice , 1998 .

[11]  P. Palffy-Muhoray,et al.  Curvature-driven foam coarsening on a sphere: A computer simulation. , 2016, Physical review. E.

[12]  Wotao Yin,et al.  A feasible method for optimization with orthogonality constraints , 2013, Math. Program..

[13]  L. Qi,et al.  Conditions for Strong Ellipticity of Anisotropic Elastic Materials , 2009 .

[14]  Juan C. Meza,et al.  On the Use of Direct Search Methods for the Molecular Conformation Problem , 1994 .

[15]  Katya Scheinberg,et al.  Geometry of interpolation sets in derivative free optimization , 2007, Math. Program..

[16]  Gene H. Golub,et al.  Matrix computations , 1983 .

[17]  Liqun Qi,et al.  Computing Eigenvalues of Large Scale Sparse Tensors Arising from a Hypergraph , 2016, SIAM J. Sci. Comput..

[18]  Katya Scheinberg,et al.  Introduction to derivative-free optimization , 2010, Math. Comput..

[19]  Philippe L. Toint,et al.  A derivative-free trust-funnel method for equality-constrained nonlinear optimization , 2014, Computational Optimization and Applications.

[20]  Ya-Xiang Yuan,et al.  Optimization theory and methods , 2006 .

[21]  R. Oeuvray,et al.  A New Derivative-Free Algorithm for the Medical Image Registration Problem , 2007 .

[22]  M. Powell.  A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation , 1994 .

[23]  Katya Scheinberg,et al.  On the convergence of derivative-free methods for unconstrained optimization , 1997 .

[24]  Richard G. Baraniuk,et al.  1-Bit compressive sensing , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[25]  D. Winfield,et al.  Function Minimization by Interpolation in a Data Table , 1973 .

[26]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.

[27]  Katya Scheinberg,et al.  Global Convergence of General Derivative-Free Trust-Region Algorithms to First- and Second-Order Critical Points , 2009, SIAM J. Optim..

[28]  Katya Scheinberg,et al.  On the local convergence of a derivative-free algorithm for least-squares minimization , 2010, Computational Optimization and Applications.

[29]  M. Powell.  The NEWUOA software for unconstrained optimization without derivatives , 2006 .

[30]  Qun Wang,et al.  Computing Extreme Eigenvalues of Large Scale Hankel Tensors , 2016, J. Sci. Comput..

[31]  Tamara G. Kolda,et al.  Optimizing an Empirical Scoring Function for Transmembrane Protein Structure Determination , 2004, INFORMS J. Comput..

[32]  Zaikun Zhang,et al.  Sobolev seminorm of quadratic functions with applications to derivative-free optimization , 2011, Math. Program..

[33]  John E. Dennis,et al.  Multidirectional search: a direct search algorithm for parallel machines , 1989 .

[34]  P. Toint,et al.  An Algorithm using Quadratic Interpolation for Unconstrained Derivative Free Optimization , 1996 .

[35]  L. Cooper,et al.  Optimal location on a sphere , 1980 .

[36]  Wotao Yin,et al.  Trust, But Verify: Fast and Accurate Signal Recovery From 1-Bit Compressive Measurements , 2011, IEEE Transactions on Signal Processing.

[37]  Elizabeth W. Karas,et al.  A trust-region derivative-free algorithm for constrained optimization , 2015, Optim. Methods Softw..

[38]  I. D. Coope,et al.  A Convergent Variant of the Nelder–Mead Algorithm , 2002 .

[39]  Liqun Qi,et al.  Eigenvalues of a real supersymmetric tensor , 2005, J. Symb. Comput..

[40]  Charles Audet,et al.  Derivative-Free and Blackbox Optimization , 2017 .

[41]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[42]  Marco Sciandrone,et al.  Sequential Penalty Derivative-Free Methods for Nonlinear Constrained Optimization , 2010, SIAM J. Optim..

[43]  Virginia Torczon,et al.  On the Convergence of Pattern Search Algorithms , 1997, SIAM J. Optim..

[44]  M. J. D. Powell,et al.  Least Frobenius norm updating of quadratic models that satisfy interpolation conditions , 2004, Math. Program..