Recourse-based stochastic nonlinear programming: properties and Benders-SQP algorithms

In this paper, we study recourse-based stochastic nonlinear programs and make two sets of contributions. The first set assumes general probability spaces and provides a deeper understanding of feasibility and recourse in stochastic nonlinear programs. A sufficient condition, for equality between the sets of feasible first-stage decisions arising from two different interpretations of almost sure feasibility, is provided. This condition is an extension to nonlinear settings of the “W-condition,” first suggested by Walkup and Wets (SIAM J. Appl. Math. 15:1299–1314, 1967). Notions of complete and relatively-complete recourse for nonlinear stochastic programs are defined and simple sufficient conditions for these to hold are given. Implications of these results on the L-shaped method are discussed. Our second set of contributions lies in the construction of a scalable, superlinearly convergent method for solving this class of problems, under the setting of a finite sample-space. We present a novel hybrid algorithm that combines sequential quadratic programming (SQP) and Benders decomposition. In this framework, the resulting quadratic programming approximations while arbitrarily large, are observed to be two-period stochastic quadratic programs (QPs) and are solved through two variants of Benders decomposition. The first is based on an inexact-cut L-shaped method for stochastic quadratic programming while the second is a quadratic extension to a trust-region method suggested by Linderoth and Wright (Comput. Optim. Appl. 24:207–250, 2003). Obtaining Lagrange multiplier estimates in this framework poses a unique challenge and are shown to be cheaply obtainable through the solution of a single low-dimensional QP. Globalization of the method is achieved through a parallelizable linesearch procedure. Finally, the efficiency and scalability of the algorithm are demonstrated on a set of stochastic nonlinear programming test problems.

[1]  Golbon Zakeri,et al.  Inexact Cuts in Benders Decomposition , 1999, SIAM J. Optim..

[2]  Gongyun Zhao,et al.  A Decomposition Method Based on SQP for a Class of Multistage Stochastic Nonlinear Programs , 2003, SIAM J. Optim..

[3]  Klaus Schittkowski,et al.  More test examples for nonlinear programming codes , 1981 .

[4]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[5]  Stephen M. Robinson,et al.  Analysis of Sample-Path Optimization , 1996, Math. Oper. Res..

[6]  Gerd Infanger,et al.  Monte Carlo (importance) sampling within a benders decomposition algorithm for stochastic linear programs , 1991, Ann. Oper. Res..

[7]  Sven Leyffer,et al.  User manual for filterSQP , 1998 .

[8]  R. Rockafellar,et al.  Stochastic convex programming: Kuhn-Tucker conditions , 1975 .

[9]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[10]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[11]  Philip E. Gill,et al.  Practical optimization , 1981 .

[12]  Maurice Queyranne,et al.  Single-Machine Scheduling Polyhedra with Precedence Constraints , 1991, Math. Oper. Res..

[13]  Francisco J. Prieto,et al.  A Sequential Quadratic Programming Algorithm Using an Incomplete Solution of the Subproblem , 1995, SIAM J. Optim..

[14]  R. Wets,et al.  STOCHASTIC PROGRAMS WITH RECOURSE , 1967 .

[15]  M. J. D. Powell,et al.  Algorithms for nonlinear constraints that use lagrangian functions , 1978, Math. Program..

[16]  Shih-Ping Han,et al.  Superlinearly convergent variable metric algorithms for general nonlinear programming problems , 1976, Math. Program..

[17]  Gongyun Zhao,et al.  A Lagrangian Dual Method with Self-Concordant Barriers for Multi-Stage Stochastic Convex Programming , 2005, Math. Program..

[18]  Xiaojun Chen,et al.  Robust solution of monotone stochastic linear complementarity problems , 2008, Math. Program..

[19]  G. Dantzig,et al.  Large-Scale Stochastic Linear Programs: Importance Sampling and Benders Decomposition , 1991 .

[20]  Roger J.-B. Wets,et al.  Continuity of some convex-cone-valued mappings , 1967 .

[21]  Tito Homem-de-Mello,et al.  Variable-sample methods for stochastic optimization , 2003, TOMC.

[22]  G. Hoek Asymptotic properties of reduction methods applying linearly equality constrained reduced problems , 1982 .

[23]  J. J. Moré,et al.  A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods , 1973 .

[24]  P. Boggs,et al.  On the Local Convergence of Quasi-Newton Methods for Constrained Optimization , 1982 .

[25]  Jörgen Blomvall,et al.  A Riccati-based primal interior point solver for multistage stochastic programming , 2002, Eur. J. Oper. Res..

[26]  Gerd Infanger Planning under uncertainty , 1992 .

[27]  Jörgen Blomvall A multistage stochastic programming algorithm suitable for parallel computing , 2003, Parallel Comput..

[28]  T. Ralphs,et al.  Decomposition Methods , 2010 .

[29]  Angelo Lucia An Explicit Quasi-Newton Update for Sparse Optimization Calculations , 1983 .

[30]  A. Shapiro Monte Carlo Sampling Methods , 2003 .

[31]  Xiaojun Chen,et al.  Random test problems and parallel methods for quadratic programs and quadratic stochastic programs , 2000 .

[32]  George B. Dantzig,et al.  Linear Programming Under Uncertainty , 2004, Manag. Sci..

[33]  John R. Birge,et al.  Decomposition and Partitioning Methods for Multistage Stochastic Linear Programs , 1985, Oper. Res..

[34]  Shuzhong Zhang,et al.  A Primal-Dual Decomposition-Based Interior Point Approach to Two-Stage Stochastic Linear Programming , 1999, Oper. Res..

[35]  Julia L. Higle,et al.  Stochastic Decomposition: A Statistical Method for Large Scale Stochastic Linear Programming , 1996 .

[36]  Klaus Schittkowski,et al.  Test examples for nonlinear programming codes , 1980 .

[37]  W. Dorn Duality in Quadratic Programming... , 2011 .

[38]  Michael J. Metternich,et al.  Planning under Uncertainty , 2004 .

[39]  Michael A. Saunders,et al.  SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization , 2002, SIAM J. Optim..

[40]  Richard A. Tapia,et al.  On the Characterization of Q-superlinear Convergence of Quasi-Newton Methods for Constrained Optimization , 1987 .

[41]  Alexander Shapiro,et al.  A simulation-based approach to two-stage stochastic programming with recourse , 1998, Math. Program..

[42]  Walter Murray,et al.  Sequential quadratic programming methods based on indefinite Hessian approximations , 1999 .

[43]  D. Shanno On variable-metric methods for sparse Hessians , 1980 .

[44]  Richard W. Cottle,et al.  Linear Complementarity Problem , 2009, Encyclopedia of Optimization.

[45]  Gongyun Zhao,et al.  A Log-Barrier method with Benders decomposition for solving two-stage stochastic linear programs , 2001, Math. Program..

[46]  Alexander Shapiro,et al.  The empirical behavior of sampling methods for stochastic programming , 2006, Ann. Oper. Res..

[47]  M. J. D. Powell,et al.  A fast algorithm for nonlinearly constrained optimization calculations , 1978 .

[48]  Paul Glasserman,et al.  Smoothing complements and randomized score functions , 1992, Ann. Oper. Res..

[49]  Michael C. Ferris,et al.  Variable-Number Sample-Path Optimization , 2008, Math. Program..

[50]  Jorge Nocedal,et al.  An Interior Point Algorithm for Large-Scale Nonlinear Programming , 1999, SIAM J. Optim..

[51]  Zdzisław Denkowski,et al.  Set-Valued Analysis , 2021 .

[52]  Peter W. Glynn,et al.  A Complementarity Framework for Forward Contracting Under Uncertainty , 2011, Oper. Res..

[53]  Krzysztof C. Kiwiel,et al.  Proximity control in bundle methods for convex nondifferentiable minimization , 1990, Math. Program..

[54]  Julia L. Higle,et al.  Inexact subgradient methods with applications in stochastic programming , 1994, Math. Program..

[55]  Paul T. Boggs,et al.  Sequential Quadratic Programming , 1995, Acta Numerica.

[56]  R. Wets,et al.  L-SHAPED LINEAR PROGRAMS WITH APPLICATIONS TO OPTIMAL CONTROL AND STOCHASTIC PROGRAMMING. , 1969 .

[57]  Gui-Hua Lin,et al.  New reformulations for stochastic nonlinear complementarity problems , 2006, Optim. Methods Softw..

[58]  Michael A. Saunders,et al.  User''s guide for NPSOL (Ver-sion 4.0): A FORTRAN package for nonlinear programming , 1984 .

[59]  Stephen J. Wright,et al.  Decomposition Algorithms for Stochastic Programming on a Computational Grid , 2001, Comput. Optim. Appl..

[60]  Antonio Alonso Ayuso,et al.  Introduction to Stochastic Programming , 2009 .

[61]  W. Marsden I and J , 2012 .

[62]  P. Toint On sparse and symmetric matrix updating subject to a linear equation , 1977 .

[63]  Gui-Hua Lin,et al.  SMOOTHING IMPLICIT PROGRAMMING APPROACHES FOR STOCHASTIC MATHEMATICAL PROGRAMS WITH LINEAR COMPLEMENTARITY CONSTRAINTS , 2003 .

[64]  Xiaojun Chen,et al.  Expected Residual Minimization Method for Stochastic Linear Complementarity Problems , 2005, Math. Oper. Res..

[65]  George B. Dantzig,et al.  Parallel processors for planning under uncertainty , 1990 .

[66]  R. Tyrrell Rockafellar,et al.  Scenarios and Policy Aggregation in Optimization Under Uncertainty , 1991, Math. Oper. Res..

[67]  Jacques F. Benders,et al.  Partitioning procedures for solving mixed-variables programming problems , 2005, Comput. Manag. Sci..

[68]  Shih-Ping Han A globally convergent method for nonlinear programming , 1975 .

[69]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[70]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[71]  Peter Kall,et al.  Stochastic Programming , 1995 .

[72]  S. M. Robinson,et al.  A quadratically-convergent algorithm for general nonlinear programming problems , 1972, Math. Program..

[73]  E. Beale ON MINIMIZING A CONVEX FUNCTION SUBJECT TO LINEAR INEQUALITIES , 1955 .

[74]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .