A stochastic approximation method for approximating the efficient frontier of chance-constrained nonlinear programs

We propose a stochastic approximation method for approximating the efficient frontier of chance-constrained nonlinear programs. Our approach is based on a bi-objective viewpoint of chance-constrained programs that seeks solutions on the efficient frontier of optimal objective value versus risk of constraint violation. To this end, we construct a reformulated problem whose objective is to minimize the probability of constraints violation subject to deterministic convex constraints (which includes a bound on the objective function value). We adapt existing smoothing-based approaches for chance-constrained problems to derive a convergent sequence of smooth approximations of our reformulated problem, and apply a projected stochastic subgradient algorithm to solve it. In contrast with exterior sampling-based approaches (such as sample average approximation) that approximate the original chance-constrained program with one having finite support, our proposal converges to stationary solutions of a smooth approximation of the original problem, thereby avoiding poor local solutions that may be an artefact of a fixed sample. Our proposal also includes a tailored implementation of the smoothing-based approach that chooses key algorithmic parameters based on problem data. Computational results on four test problems from the literature indicate that our proposed approach can efficiently determine good approximations of the efficient frontier.

[1]  Alexander Shapiro,et al.  Convex Approximations of Chance Constrained Programs , 2006, SIAM J. Optim..

[2]  Zhiqiang Zhou,et al.  Algorithms for stochastic optimization with function or expectation constraints , 2016, Comput. Optim. Appl..

[3]  Shiqian Ma,et al.  Penalty methods with stochastic approximation for stochastic nonlinear programming , 2013, Math. Comput..

[4]  G. Cohen,et al.  Stochastic Programming with Probability , 2007, 0708.0281.

[5]  Benjamin Müller,et al.  The SCIP Optimization Suite 5.0 , 2017, 2112.08872.

[6]  WächterAndreas,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006 .

[7]  Xiang Li,et al.  Probabilistically Constrained Linear Programs and Risk-Adjusted Controller Design , 2005, SIAM J. Optim..

[8]  René Henrion,et al.  Gradient Formulae for Nonlinear Probabilistic Constraints with Gaussian and Gaussian-Like Distributions , 2014, SIAM J. Optim..

[9]  Dmitriy Drusvyatskiy,et al.  Stochastic Subgradient Method Converges on Tame Functions , 2018, Foundations of Computational Mathematics.

[10]  Claudia A. Sagastizábal,et al.  Probabilistic optimization via approximate p-efficient points and bundle methods , 2017, Comput. Oper. Res..

[11]  András Prékopa,et al.  ON PROBABILISTIC CONSTRAINED PROGRAMMING , 2015 .

[12]  Melvyn Sim,et al.  From CVaR to Uncertainty Set: Implications in Joint Chance-Constrained Optimization , 2010, Oper. Res..

[13]  Saeed Ghadimi,et al.  Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization , 2013, Mathematical Programming.

[14]  Riho Lepp.  Extremum Problems with Probability Functions: Kernel Type Solution Methods , 2009, Encyclopedia of Optimization.

[15]  Michael Chertkov,et al.  Chance-Constrained Optimal Power Flow: Risk-Aware Network Control under Uncertainty , 2012, SIAM Rev..

[16]  Abebe Geletu,et al.  An Inner-Outer Approximation Approach to Chance Constrained Optimization , 2017, SIAM J. Optim..

[17]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[18]  René Henrion,et al.  Solving joint chance constrained problems using regularization and Benders’ decomposition , 2018, Annals of Operations Research.

[19]  Iain Dunning,et al.  JuMP: A Modeling Language for Mathematical Optimization , 2015, SIAM Rev..

[20]  R. Jagannathan,et al.  Chance-Constrained Programming with Joint Constraints , 1974, Oper. Res..

[21]  R. Wets,et al.  Stochastic programming , 1989 .

[22]  Dmitriy Drusvyatskiy,et al.  Efficiency of minimizing compositions of convex functions and smooth maps , 2016, Math. Program..

[23]  H. Ruben,et al.  Probability Content of Regions Under Spherical Normal Distributions, IV: The Distribution of Homogeneous and Non-Homogeneous Quadratic Functions of Normal Variables , 1961 .

[24]  Mengdi Wang,et al.  Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions , 2014, Mathematical Programming.

[25]  Yuri M. Ermoliev.  Stochastic Quasigradient Methods , 2009, Encyclopedia of Optimization.

[26]  Alexander Shapiro,et al.  Stochastic Approximation approach to Stochastic Programming , 2013 .

[27]  Pu Li,et al.  Chance constrained programming approach to process optimization under uncertainty , 2008, Comput. Chem. Eng..

[28]  Mingrui Liu,et al.  Non-Convex Min-Max Optimization: Provable Algorithms and Applications in Machine Learning , 2018, ArXiv.

[29]  Niao He,et al.  On the Convergence Rate of Stochastic Mirror Descent for Nonsmooth Nonconvex Optimization , 2018, 1806.04781.

[30]  Giuseppe Carlo Calafiore,et al.  Uncertain convex programs: randomized solutions and confidence levels , 2005, Math. Program..

[31]  F. Vázquez-Abad,et al.  Stochastic Programming with Probability Constraints , 2007 .

[32]  V. Zavala,et al.  A Sigmoidal Approximation for Chance-Constrained Nonlinear Programs , 2020, 2004.02402.

[33]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[34]  James R. Luedtke,et al.  A Sample Approximation Approach for Optimization with Probabilistic Constraints , 2008, SIAM J. Optim..

[35]  Alexander Shapiro,et al.  Sample Average Approximation Method for Chance Constrained Programming: Theory and Applications , 2009, J. Optimization Theory and Applications.

[36]  V. Norkin.  The Analysis and Optimization of Probability Functions , 1993 .

[37]  E. A. Nurminskii.  The quasigradient method for the solving of the nonlinear programming problems , 1973 .

[38]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[39]  Y. Ermoliev,et al.  The Minimization of Semicontinuous Functions: Mollifier Subgradients , 1995 .

[40]  Antonio Frangioni,et al.  Inexact stabilized Benders’ decomposition approaches with application to chance-constrained problems with finite support , 2016, Comput. Optim. Appl..

[41]  René Henrion,et al.  (Sub-)Gradient Formulae for Probability Functions of Random Inequality Systems under Gaussian Distribution , 2017, SIAM/ASA J. Uncertain. Quantification.

[42]  David P. Morton,et al.  Estimating the efficient frontier of a probabilistic bicriteria model , 2009, Proceedings of the 2009 Winter Simulation Conference (WSC).

[43]  Laurent Condat,et al.  A Fast Projection onto the Simplex and the l 1 Ball , 2015 .

[44]  V. Norkin,et al.  Stochastic generalized gradient method for nonconvex nonsmooth stochastic optimization , 1998 .

[45]  Alexander Shapiro,et al.  Lectures on Stochastic Programming: Modeling and Theory , 2009 .

[46]  James R. Luedtke.  A branch-and-cut decomposition algorithm for solving chance-constrained mathematical programs with finite support , 2013, Mathematical Programming.

[47]  Martin Branda,et al.  Nonlinear Chance Constrained Problems: Optimality Conditions, Regularization and Solvers , 2016, Journal of Optimization Theory and Applications.

[48]  René Henrion,et al.  On the quantification of nomination feasibility in stationary gas networks with random load , 2016, Math. Methods Oper. Res..

[49]  James R. Luedtke,et al.  Solving Chance-Constrained Problems via a Smooth Sample-Based Nonlinear Approximation , 2019, SIAM J. Optim..

[50]  Xiantao Xiao,et al.  Convergence analysis on a smoothing approach to joint chance constrained programs , 2016 .

[51]  Hui Zhang,et al.  Chance Constrained Programming for Optimal Power Flow Under Uncertainty , 2011, IEEE Transactions on Power Systems.

[52]  Jacques F. Benders,et al.  Partitioning procedures for solving mixed-variables programming problems , 2005, Comput. Manag. Sci..

[53]  A. Charnes,et al.  Cost Horizons and Certainty Equivalents: An Approach to Stochastic Programming of Heating Oil , 1958 .

[54]  Ruiwei Jiang,et al.  Data-driven chance constrained stochastic program , 2015, Mathematical Programming.

[55]  Patrick Amestoy,et al.  MUMPS : A General Purpose Distributed Memory Sparse Solver , 2000, PARA.

[56]  Martin Branda,et al.  Machine learning approach to chance-constrained problems: An algorithm based on the stochastic gradient descent , 2019, 1905.10986.

[57]  E. Polak,et al.  Reliability-based optimal design using sample average approximations , 2004 .

[58]  Alan Edelman,et al.  Julia: A Fresh Approach to Numerical Computing , 2014, SIAM Rev..

[59]  Marco C. Campi,et al.  A Sampling-and-Discarding Approach to Chance-Constrained Optimization: Feasibility and Optimality , 2011, J. Optim. Theory Appl..

[60]  Dmitriy Drusvyatskiy,et al.  Stochastic subgradient method converges at the rate O(k-1/4) on weakly convex functions , 2018, ArXiv.

[61]  Giuseppe Carlo Calafiore,et al.  Research on probabilistic methods for control system design , 2011, Autom..

[62]  Maria Gabriela Martinez,et al.  Regularization methods for optimization problems with probabilistic constraints , 2013, Math. Program..

[63]  Michael I. Jordan,et al.  On the Local Minima of the Empirical Risk , 2018, NeurIPS.

[64]  Laurent El Ghaoui,et al.  Robust Optimization , 2021, ICORES.

[65]  Patrick L. Combettes,et al.  On the effectiveness of projection methods for convex feasibility problems with linear inequality constraints , 2009, Computational Optimization and Applications.

[66]  Victor M. Zavala,et al.  A Sequential Algorithm for Solving Nonlinear Optimization Problems with Chance Constraints , 2018, SIAM J. Optim..

[67]  R. Rockafellar,et al.  Optimization of conditional value-at risk , 2000 .

[68]  Yuri Ermoliev,et al.  On nonsmooth and discontinuous problems of stochastic systems optimization , 1997 .

[69]  Yi Yang,et al.  Sequential Convex Approximations to Joint Chance Constrained Programs: A Monte Carlo Approach , 2011, Oper. Res..

[70]  Liwei Zhang,et al.  A Smoothing Function Approach to Joint Chance-Constrained Programs , 2014, J. Optim. Theory Appl..

[71]  L. J. Hong,et al.  A smooth Monte Carlo approach to joint chance-constrained programs , 2013 .