Low Order-Value Optimization and applications

Given r real functions F1(x),...,Fr(x) and an integer p between 1 and r, the Low Order-Value Optimization problem (LOVO) consists of minimizing the sum of the functions that take the p smaller values. If (y1,...,yr) is a vector of data and T(x, ti) is the predicted value of the observation i with the parameters $$x \in I\!\!R^n$$ , it is natural to define Fi(x) =  (T(x, ti) − yi)2 (the quadratic error in observation i under the parameters x). When p =  r this LOVO problem coincides with the classical nonlinear least-squares problem. However, the interesting situation is when p is smaller than r. In that case, the solution of LOVO allows one to discard the influence of an estimated number of outliers. Thus, the LOVO problem is an interesting tool for robust estimation of parameters of nonlinear models. When p ≪ r the LOVO problem may be used to find hidden structures in data sets. One of the most successful applications includes the Protein Alignment problem. Fully documented algorithms for this application are available at www.ime.unicamp.br/~martinez/lovoalign. In this paper optimality conditions are discussed, algorithms for solving the LOVO problem are introduced and convergence theorems are proved. Finally, numerical experiments are presented.

[1]  Phhilippe Jorion.  Value at Risk: The New Benchmark for Managing Financial Risk , 2000 .

[2]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[3]  M. Levitt,et al.  Structural similarity of DNA-binding domains of bacteriophage repressors and the globin core , 1993, Current Biology.

[4]  José Mario Martínez,et al.  Continuous optimization methods for structure alignments , 2007, Math. Program..

[5]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[6]  R. Andreani,et al.  On the Relation between Constant Positive Linear Dependence Condition and Quasinormality Constraint Qualification , 2005 .

[7]  T. N. Bhat,et al.  The Protein Data Bank , 2000, Nucleic Acids Res..

[8]  R. Rockafellar.  Augmented Lagrange Multiplier Functions and Duality in Nonconvex Programming , 1974 .

[9]  Rachel Kolodny,et al.  Comprehensive evaluation of protein structure alignment methods: scoring by geometric measures. , 2005, Journal of molecular biology.

[10]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[11]  Zdenek Dostál,et al.  Augmented Lagrangians with Adaptive Precision Control for Quadratic Programming with Simple Bounds and Equality Constraints , 2002, SIAM J. Optim..

[12]  José Mario Martínez,et al.  Global minimization using an Augmented Lagrangian method with variable lower-level constraints , 2010, Math. Program..

[13]  S. Kearsley.  On the orthogonal transformation used for structural comparisons , 1989 .

[14]  R. Rockafellar,et al.  Conditional Value-at-Risk for General Loss Distributions , 2001 .

[15]  José Mario Martínez,et al.  Augmented Lagrangian methods under the constant positive linear dependence constraint qualification , 2007, Math. Program..

[16]  W. Hager.  Analysis and implementation of a dual algorithm for constrained optimization , 1993 .

[17]  José Mario Martínez,et al.  Order-Value Optimization: Formulation and solution by means of a primal cauchy method , 2003, Math. Methods Oper. Res..

[18]  Adrian S. Lewis,et al.  A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization , 2005, SIAM J. Optim..

[19]  José Mario Martínez,et al.  Convergent algorithms for protein structural alignment , 2007, BMC Bioinformatics.

[20]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[21]  Le Thi Hoai An,et al.  A D.C. Optimization Algorithm for Solving the Trust-Region Subproblem , 1998, SIAM J. Optim..

[22]  J. M. Martínez,et al.  Quasi-Newton methods for Order-value optimization and value-at-risk calculations , 2006 .

[23]  José Mario Martínez,et al.  Order-value optimization and new applications , 2009 .

[24]  T. P. Dinh,et al.  Convex analysis approach to d.c. programming: Theory, Algorithm and Applications , 1997 .

[25]  José Mario Martínez,et al.  Nonlinear-programming reformulation of the order-value optimization problem , 2005, Math. Methods Oper. Res..

[26]  R. Tyrrell Rockafellar,et al.  Lagrange Multipliers and Optimality , 1993, SIAM Rev..

[27]  Z. Dostál,et al.  Duality-based domain decomposition with natural coarse-space for variational inequalities0 , 2000 .

[28]  José Mario Martínez,et al.  Large-Scale Active-Set Box-Constrained Optimization Method with Spectral Projected Gradients , 2002, Comput. Optim. Appl..

[29]  Nathan Linial,et al.  Approximate protein structural alignment in polynomial time. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[30]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[31]  Zengxin Wei,et al.  On the Constant Positive Linear Dependence Condition and Its Application to SQP Methods , 1999, SIAM J. Optim..

[32]  W. Kabsch.  A discussion of the solution for the best rotation to relate two sets of vectors , 1978 .

[33]  M. J. D. Powell,et al.  A method for nonlinear constraints in minimization problems , 1969 .

[34]  R. Rockafellar.  The multiplier method of Hestenes and Powell applied to convex programming , 1973 .

[35]  C Sander,et al.  Mapping the Protein Universe , 1996, Science.

[36]  Zdeněk Dostál,et al.  Inexact Semimonotonic Augmented Lagrangians with Optimal Feasibility Convergence for Convex Bound and Equality Constrained Quadratic Programming , 2005, SIAM J. Numer. Anal..

[37]  José Mario Martínez,et al.  On Augmented Lagrangian Methods with General Lower-Level Constraints , 2007, SIAM J. Optim..

[38]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[39]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[40]  Nicholas I. M. Gould,et al.  Convergence Properties of an Augmented Lagrangian Algorithm for Optimization with a Combination of General Equality and Linear Constraints , 1996, SIAM J. Optim..

[41]  Charles Audet,et al.  Mesh Adaptive Direct Search Algorithms for Constrained Optimization , 2006, SIAM J. Optim..

[42]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[43]  P. Toint,et al.  A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds , 1991 .

[44]  O. Mangasarian,et al.  The Fritz John Necessary Optimality Conditions in the Presence of Equality and Inequality Constraints , 1967 .

[45]  Roberto Andreani,et al.  Global Order-Value Optimization by means of a Multistart Harmonic Oscillator Tunneling Strategy , 2006 .

[46]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .