A stochastic approximation method for approximating the efficient frontier of chance-constrained nonlinear programs

We propose a stochastic approximation method for approximating the efficient frontier of chance-constrained nonlinear programs. Our approach is based on a bi-objective viewpoint of chance-constrained programs that seeks solutions on the efficient frontier of optimal objective value versus risk of constraint violation. To this end, we construct a reformulated problem whose objective is to minimize the probability of constraints violation subject to deterministic convex constraints (which includes a bound on the objective function value). We adapt existing smoothing-based approaches for chance-constrained problems to derive a convergent sequence of smooth approximations of our reformulated problem, and apply a projected stochastic subgradient algorithm to solve it. In contrast with exterior sampling-based approaches (such as sample average approximation) that approximate the original chance-constrained program with one having finite support, our proposal converges to stationary solutions of a smooth approximation of the original problem, thereby avoiding poor local solutions that may be an artefact of a fixed sample. Our proposal also includes a tailored implementation of the smoothing-based approach that chooses key algorithmic parameters based on problem data. Computational results on four test problems from the literature indicate that our proposed approach can efficiently determine good approximations of the efficient frontier.

[1]  Giuseppe Carlo Calafiore,et al.  Research on probabilistic methods for control system design , 2011, Autom..

[2]  G. Cohen,et al.  Stochastic Programming with Probability , 2007, 0708.0281.

[3]  René Henrion,et al.  On the quantification of nomination feasibility in stationary gas networks with random load , 2016, Math. Methods Oper. Res..

[4]  Y. Ermoliev,et al.  The Minimization of Semicontinuous Functions: Mollifier Subgradients , 1995 .

[5]  Laurent Condat.  Fast projection onto the simplex and the l1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pmb {l}_\mathbf {1}$$\end{ , 2015, Mathematical Programming.

[6]  Patrick Amestoy,et al.  MUMPS : A General Purpose Distributed Memory Sparse Solver , 2000, PARA.

[7]  Alan Edelman,et al.  Julia: A Fresh Approach to Numerical Computing , 2014, SIAM Rev..

[8]  L. J. Hong,et al.  A smooth Monte Carlo approach to joint chance-constrained programs , 2013 .

[9]  James R. Luedtke,et al.  Solving Chance-Constrained Problems via a Smooth Sample-Based Nonlinear Approximation , 2019, SIAM J. Optim..

[10]  Giuseppe Carlo Calafiore,et al.  Uncertain convex programs: randomized solutions and confidence levels , 2005, Math. Program..

[11]  Riho Lepp.  Extremum Problems with Probability Functions: Kernel Type Solution Methods , 2009, Encyclopedia of Optimization.

[12]  Abebe Geletu,et al.  An Inner-Outer Approximation Approach to Chance Constrained Optimization , 2017, SIAM J. Optim..

[13]  Liwei Zhang,et al.  A Smoothing Function Approach to Joint Chance-Constrained Programs , 2014, J. Optim. Theory Appl..

[14]  Dmitriy Drusvyatskiy,et al.  Stochastic Subgradient Method Converges on Tame Functions , 2018, Foundations of Computational Mathematics.

[15]  James R. Luedtke,et al.  A Sample Approximation Approach for Optimization with Probabilistic Constraints , 2008, SIAM J. Optim..

[16]  Maria Gabriela Martinez,et al.  Regularization methods for optimization problems with probabilistic constraints , 2013, Math. Program..

[17]  Peter Kall,et al.  Stochastic Programming , 1995 .

[18]  Xiang Li,et al.  Probabilistically Constrained Linear Programs and Risk-Adjusted Controller Design , 2005, SIAM J. Optim..

[19]  R. Jagannathan,et al.  Chance-Constrained Programming with Joint Constraints , 1974, Oper. Res..

[20]  Alexander Shapiro,et al.  Convex Approximations of Chance Constrained Programs , 2006, SIAM J. Optim..

[21]  René Henrion,et al.  Gradient Formulae for Nonlinear Probabilistic Constraints with Gaussian and Gaussian-Like Distributions , 2014, SIAM J. Optim..

[22]  Michael Chertkov,et al.  Chance-Constrained Optimal Power Flow: Risk-Aware Network Control under Uncertainty , 2012, SIAM Rev..

[23]  H. Ruben,et al.  Probability Content of Regions Under Spherical Normal Distributions, IV: The Distribution of Homogeneous and Non-Homogeneous Quadratic Functions of Normal Variables , 1961 .

[24]  Marco C. Campi,et al.  A Sampling-and-Discarding Approach to Chance-Constrained Optimization: Feasibility and Optimality , 2011, J. Optim. Theory Appl..

[25]  A. Charnes,et al.  Cost Horizons and Certainty Equivalents: An Approach to Stochastic Programming of Heating Oil , 1958 .

[26]  Dmitriy Drusvyatskiy,et al.  Stochastic subgradient method converges at the rate O(k-1/4) on weakly convex functions , 2018, ArXiv.

[27]  David P. Morton,et al.  Estimating the efficient frontier of a probabilistic bicriteria model , 2009, Proceedings of the 2009 Winter Simulation Conference (WSC).

[28]  C. Blair.  Problem Complexity and Method Efficiency in Optimization (A. S. Nemirovsky and D. B. Yudin) , 1985 .

[29]  Alexander Shapiro,et al.  Stochastic Approximation approach to Stochastic Programming , 2013 .

[30]  J. F. Benders.  Partitioning procedures for solving mixed-variables programming problems , 1962 .

[31]  Victor M. Zavala,et al.  A Sequential Algorithm for Solving Nonlinear Optimization Problems with Chance Constraints , 2018, SIAM J. Optim..

[32]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[33]  Melvyn Sim,et al.  From CVaR to Uncertainty Set: Implications in Joint Chance-Constrained Optimization , 2010, Oper. Res..

[34]  Mingrui Liu,et al.  Non-Convex Min-Max Optimization: Provable Algorithms and Applications in Machine Learning , 2018, ArXiv.

[35]  E. A. Nurminskii.  The quasigradient method for the solving of the nonlinear programming problems , 1973 .

[36]  Antonio Frangioni,et al.  Inexact stabilized Benders’ decomposition approaches with application to chance-constrained problems with finite support , 2016, Comput. Optim. Appl..

[37]  Ruiwei Jiang,et al.  Data-driven chance constrained stochastic program , 2015, Mathematical Programming.

[38]  Laurent Condat,et al.  A Fast Projection onto the Simplex and the l 1 Ball , 2015 .

[39]  V. Zavala,et al.  A Sigmoidal Approximation for Chance-Constrained Nonlinear Programs , 2020, 2004.02402.

[40]  Michael I. Jordan,et al.  On the Local Minima of the Empirical Risk , 2018, NeurIPS.

[41]  Niao He,et al.  On the Convergence Rate of Stochastic Mirror Descent for Nonsmooth Nonconvex Optimization , 2018, 1806.04781.

[42]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[43]  Martin Branda,et al.  Machine learning approach to chance-constrained problems: An algorithm based on the stochastic gradient descent , 2019, 1905.10986.

[44]  Yi Yang,et al.  Sequential Convex Approximations to Joint Chance Constrained Programs: A Monte Carlo Approach , 2011, Oper. Res..

[45]  Patrick L. Combettes,et al.  On the effectiveness of projection methods for convex feasibility problems with linear inequality constraints , 2009, Computational Optimization and Applications.

[46]  Martin Branda,et al.  Nonlinear Chance Constrained Problems: Optimality Conditions, Regularization and Solvers , 2016, Journal of Optimization Theory and Applications.

[47]  Xiantao Xiao,et al.  Convergence analysis on a smoothing approach to joint chance constrained programs , 2016 .

[48]  András Prékopa,et al.  ON PROBABILISTIC CONSTRAINED PROGRAMMING , 2015 .

[49]  Zhiqiang Zhou,et al.  Algorithms for stochastic optimization with function or expectation constraints , 2020, Comput. Optim. Appl..

[50]  René Henrion,et al.  (Sub-)Gradient Formulae for Probability Functions of Random Inequality Systems under Gaussian Distribution , 2017, SIAM/ASA J. Uncertain. Quantification.

[51]  R. Rockafellar,et al.  Optimization of conditional value-at risk , 2000 .

[52]  B. L. Miller,et al.  Chance Constrained Programming with Joint Constraints , 1965 .

[53]  Dmitriy Drusvyatskiy,et al.  Efficiency of minimizing compositions of convex functions and smooth maps , 2016, Math. Program..

[54]  Mengdi Wang,et al.  Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions , 2014, Mathematical Programming.

[55]  Jacques F. Benders,et al.  Partitioning procedures for solving mixed-variables programming problems , 2005, Comput. Manag. Sci..

[56]  V. Norkin.  The Analysis and Optimization of Probability Functions , 1993 .

[57]  James R. Luedtke.  A branch-and-cut decomposition algorithm for solving chance-constrained mathematical programs with finite support , 2013, Mathematical Programming.

[58]  R. Wets,et al.  Stochastic programming , 1989 .

[59]  Yuri M. Ermoliev.  Stochastic Quasigradient Methods , 2009, Encyclopedia of Optimization.

[60]  Pu Li,et al.  Chance constrained programming approach to process optimization under uncertainty , 2008, Comput. Chem. Eng..

[61]  Alexander Shapiro,et al.  Lectures on Stochastic Programming: Modeling and Theory , 2009 .

[62]  Claudia A. Sagastizábal,et al.  Probabilistic optimization via approximate p-efficient points and bundle methods , 2017, Comput. Oper. Res..

[63]  Iain Dunning,et al.  JuMP: A Modeling Language for Mathematical Optimization , 2015, SIAM Rev..

[64]  Benjamin Müller,et al.  The SCIP Optimization Suite 5.0 , 2017, 2112.08872.

[65]  Hui Zhang,et al.  Chance Constrained Programming for Optimal Power Flow Under Uncertainty , 2011, IEEE Transactions on Power Systems.

[66]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[67]  V. Norkin,et al.  Stochastic generalized gradient method for nonconvex nonsmooth stochastic optimization , 1998 .

[68]  René Henrion,et al.  Solving joint chance constrained problems using regularization and Benders’ decomposition , 2018, Annals of Operations Research.

[69]  Yuri Ermoliev,et al.  On nonsmooth and discontinuous problems of stochastic systems optimization , 1997 .

[70]  Saeed Ghadimi,et al.  Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization , 2013, Mathematical Programming.

[71]  E. Polak,et al.  Reliability-based optimal design using sample average approximations , 2004 .

[72]  Laurent El Ghaoui,et al.  Robust Optimization , 2021, ICORES.

[73]  Shiqian Ma,et al.  Penalty methods with stochastic approximation for stochastic nonlinear programming , 2013, Math. Comput..

[74]  Alexander Shapiro,et al.  Sample Average Approximation Method for Chance Constrained Programming: Theory and Applications , 2009, J. Optimization Theory and Applications.

[75]  WächterAndreas,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006 .

[76]  F. Vázquez-Abad,et al.  Stochastic Programming with Probability Constraints , 2007 .