Distributed Algorithms for Robust Convex Optimization via the Scenario Approach

This paper proposes distributed algorithms to solve robust convex optimization (RCO) when the constraints are affected by nonlinear uncertainty. We adopt a scenario approach by randomly sampling the uncertainty set. To facilitate the computational task, instead of using a single centralized processor to obtain a “global solution” of the scenario problem (SP), we resort to <italic>multiple interconnected processors</italic> that are distributed among different nodes of a network to simultaneously solve the SP. Then, we propose a primal-dual subgradient algorithm and a random projection algorithm to distributedly solve the SP over undirected and directed graphs, respectively. Both algorithms are given in an explicit recursive form with simple iterations, which are especially suited for processors with limited computational capability. We show that, if the underlying graph is strongly connected, each node asymptotically computes a common optimal solution to the SP with a convergence rate <inline-formula><tex-math notation="LaTeX">$O(1/(\sum _{t=1}^k\zeta ^t))$</tex-math></inline-formula>, where <inline-formula><tex-math notation="LaTeX">$\lbrace \zeta ^t\rbrace$</tex-math></inline-formula> is a sequence of appropriately decreasing stepsizes. That is, the RCO is effectively solved in a distributed way. The relations with the existing literature on robust convex programs are thoroughly discussed and an example of robust system identification is included to validate the effectiveness of our distributed algorithms.

[1]  Angelia Nedic,et al.  Asynchronous Gossip-Based Random Projection Algorithms Over Networks , 2013, IEEE Transactions on Automatic Control.

[2]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[3]  Damiano Varagnolo,et al.  Newton-Raphson Consensus for Distributed Convex Optimization , 2015, IEEE Transactions on Automatic Control.

[4]  Giuseppe Carlo Calafiore,et al.  Distributed Random Convex Programming via Constraints Consensus , 2012, SIAM J. Control. Optim..

[5]  Arkadi Nemirovski,et al.  Robust Convex Optimization , 1998, Math. Oper. Res..

[6]  Angelia Nedic,et al.  Random algorithms for convex minimization problems , 2011, Math. Program..

[7]  Martin J. Wainwright,et al.  Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling , 2010, IEEE Transactions on Automatic Control.

[8]  Giuseppe Carlo Calafiore,et al.  Research on probabilistic methods for control system design , 2011, Autom..

[9]  Yinyu Ye,et al.  Distributionally Robust Optimization Under Moment Uncertainty with Application to Data-Driven Problems , 2010, Oper. Res..

[10]  Usman A. Khan,et al.  Directed-Distributed Gradient Descent , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[11]  Bahman Gharesifard,et al.  Distributed Continuous-Time Convex Optimization on Weight-Balanced Digraphs , 2012, IEEE Transactions on Automatic Control.

[12]  Giuseppe Carlo Calafiore,et al.  The scenario approach to robust control design , 2006, IEEE Transactions on Automatic Control.

[13]  Shie Mannor,et al.  Robust Regression and Lasso , 2008, IEEE Transactions on Information Theory.

[14]  Laurent El Ghaoui,et al.  Robust Solutions to Least-Squares Problems with Uncertain Data , 1997, SIAM J. Matrix Anal. Appl..

[15]  Xiangfeng Wang,et al.  Multi-Agent Distributed Optimization via Inexact Consensus ADMM , 2014, IEEE Transactions on Signal Processing.

[16]  Jean-Philippe Vial,et al.  Robust Optimization , 2021, ICORES.

[17]  Giuseppe Notarstefano,et al.  Distributed Abstract Optimization via Constraints Consensus: Theory and Applications , 2011, IEEE Transactions on Automatic Control.

[18]  Darinka Dentcheva,et al.  An augmented Lagrangian method for distributed optimization , 2015, Math. Program..

[19]  Qing Ling,et al.  On the Linear Convergence of the ADMM in Decentralized Consensus Optimization , 2013, IEEE Transactions on Signal Processing.

[20]  Roberto Tempo,et al.  Randomized methods for design of uncertain systems: Sample complexity and sequential algorithms , 2013, Autom..

[21]  Mario Sznaier,et al.  Randomized Algorithms for Analysis and Control of Uncertain Systems with Applications, Second Edition, Roberto Tempo, Giuseppe Calafiore, Fabrizio Dabbene (Eds.). Springer-Verlag, London (2013), 357, ISBN: 978-1-4471-4609-4 , 2014, Autom..

[22]  Boris Polyak Random Algorithms for Solving Convex Inequalities , 2001 .

[23]  Dick den Hertog,et al.  A practical guide to robust optimization , 2015, 1501.02634.

[24]  Giuseppe Carlo Calafiore,et al.  Uncertain convex programs: randomized solutions and confidence levels , 2005, Math. Program..

[25]  Ian R. Petersen,et al.  Robust control of uncertain systems: Classical results and recent developments , 2014, Autom..

[26]  B. Ross Barmish,et al.  The uniform distribution: A rigorous justification for its use in robustness analysis , 1996, Math. Control. Signals Syst..

[27]  H. Robbins,et al.  A Convergence Theorem for Non Negative Almost Supermartingales and Some Applications , 1985 .

[28]  Marco C. Campi,et al.  The Exact Feasibility of Randomized Solutions of Uncertain Convex Programs , 2008, SIAM J. Optim..

[29]  Karl Henrik Johansson,et al.  Approximate Projected Consensus for Convex Intersection Computation: Convergence Analysis and Critical Error Angle , 2014, IEEE Transactions on Automatic Control.

[30]  A. Shapiro ON DUALITY THEORY OF CONIC LINEAR PROBLEMS , 2001 .

[31]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.

[32]  Maria Prandini,et al.  Distributed Constrained Optimization and Consensus in Uncertain Networks via Proximal Minimization , 2016, IEEE Transactions on Automatic Control.

[33]  C. W. Scherer,et al.  Relaxations for Robust Linear Matrix Inequality Problems with Verifications for Exactness , 2005, SIAM J. Matrix Anal. Appl..

[34]  Lihua Xie,et al.  Network Topology and Communication Data Rate for Consensusability of Discrete-Time Multi-Agent Systems , 2011, IEEE Trans. Autom. Control..

[35]  Angelia Nedic,et al.  Convergence Rate of Distributed Averaging Dynamics and Optimization in Networks , 2015, Found. Trends Syst. Control..

[36]  Pascal Bianchi,et al.  Explicit Convergence Rate of a Distributed Alternating Direction Method of Multipliers , 2013, IEEE Transactions on Automatic Control.

[37]  Angelia Nedic,et al.  Distributed Optimization Over Time-Varying Directed Graphs , 2015, IEEE Trans. Autom. Control..

[38]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[39]  Stephen P. Boyd,et al.  Subgradient Methods , 2007 .

[40]  Anna Scaglione,et al.  Distributed Constrained Optimization by Consensus-Based Primal-Dual Perturbation Method , 2013, IEEE Transactions on Automatic Control.

[41]  Constantine Caramanis,et al.  Theory and Applications of Robust Optimization , 2010, SIAM Rev..

[42]  Dimitri P. Bertsekas,et al.  Convex Optimization Algorithms , 2015 .

[43]  R. Tempo,et al.  Randomized Algorithms for Analysis and Control of Uncertain Systems , 2004 .