Worst-Case Risk Quantification under Distributional Ambiguity using Kernel Mean Embedding in Moment Problem

In order to anticipate rare and impactful events, we propose to quantify the worst-case risk under distributional ambiguity using a recent development in kernel methods — the kernel mean embedding. Specifically, we formulate the generalized moment problem whose ambiguity set (i.e., the moment constraint) is described by constraints in the associated reproducing kernel Hilbert space in a nonparametric manner. We then present the tractable approximation and its theoretical justification. As a concrete application, we numerically test the proposed method in characterizing the worst-case constraint violation probability in the context of a constrained stochastic control system.

[1]  O. Gaans Probability measures on metric spaces , 2022 .

[2]  Daniel Kuhn,et al.  Distributionally robust joint chance constraints with second-order moment information , 2011, Mathematical Programming.

[3]  Kenji Fukumizu,et al.  Recovering Distributions from Gaussian RKHS Embeddings , 2014, AISTATS.

[4]  John C. Duchi,et al.  Certifying Some Distributional Robustness with Principled Adversarial Training , 2017, ICLR.

[5]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .

[6]  Marshall F Chalverus,et al.  The Black Swan: The Impact of the Highly Improbable , 2007 .

[7]  Giuseppe Carlo Calafiore,et al.  The scenario approach to robust control design , 2006, IEEE Transactions on Automatic Control.

[8]  Ioana Popescu,et al.  Optimal Inequalities in Probability Theory: A Convex Optimization Approach , 2005, SIAM J. Optim..

[9]  K SriperumbudurBharath,et al.  Universality, Characteristic Kernels and RKHS Embedding of Measures , 2011 .

[10]  Yinyu Ye,et al.  Distributionally Robust Optimization Under Moment Uncertainty with Application to Data-Driven Problems , 2010, Oper. Res..

[11]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[12]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[13]  Daniel Kuhn,et al.  Generalized Gauss inequalities via semidefinite programming , 2015, Mathematical Programming.

[14]  Vishal Gupta,et al.  Data-driven robust optimization , 2013, Math. Program..

[15]  Anja De Waegenaere,et al.  Robust Solutions of Optimization Problems Affected by Uncertain Probabilities , 2011, Manag. Sci..

[16]  Le Song,et al.  A Hilbert Space Embedding for Distributions , 2007, Discovery Science.

[17]  Nassim Nicholas Taleb,et al.  The Black Swan: The Impact of the Highly Improbable , 2007 .

[18]  J. Lasserre Bounds on measures satisfying moment conditions , 2002 .

[19]  Ioana Popescu,et al.  A Semidefinite Programming Approach to Optimal-Moment Bounds for Convex Classes of Distributions , 2005, Math. Oper. Res..

[20]  Stephen P. Boyd,et al.  Generalized Chebyshev Bounds via Semidefinite Programming , 2007, SIAM Rev..

[21]  Houman Owhadi,et al.  Optimal Uncertainty Quantification , 2010, SIAM Rev..

[22]  Daniel Kuhn,et al.  Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations , 2015, Mathematical Programming.

[23]  Marcello Farina,et al.  Stochastic linear Model Predictive Control with chance constraints – A review , 2016 .

[24]  Moritz Diehl,et al.  CasADi: a software framework for nonlinear optimization and optimal control , 2018, Mathematical Programming Computation.

[25]  Daniel Kuhn,et al.  Distributionally Robust Control of Constrained Stochastic Systems , 2016, IEEE Transactions on Automatic Control.

[26]  D. Bertsimas,et al.  Moment Problems and Semidefinite Optimization , 2000 .

[27]  Melvyn Sim,et al.  Adaptive Distributionally Robust Optimization , 2019, Manag. Sci..

[28]  A. Mesbah,et al.  Stochastic Model Predictive Control: An Overview and Perspectives for Future Research , 2016, IEEE Control Systems.

[29]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[30]  Garud Iyengar,et al.  Ambiguous chance constrained problems and robust optimization , 2006, Math. Program..

[31]  A. Shapiro ON DUALITY THEORY OF CONIC LINEAR PROBLEMS , 2001 .

[32]  石井 恵一 On sharpness of Tchebycheff-type inequalities = チェビシェフ型不等式の最良性について , 1964 .

[33]  YeYinyu,et al.  Distributionally Robust Optimization Under Moment Uncertainty with Application to Data-Driven Problems , 2010 .

[34]  Kenji Fukumizu,et al.  Universality, Characteristic Kernels and RKHS Embedding of Measures , 2010, J. Mach. Learn. Res..

[35]  Bernhard Schölkopf,et al.  A Kernel Mean Embedding Approach to Reducing Conservativeness in Stochastic Programming and Control , 2020, L4DC.

[36]  Francis R. Bach,et al.  On the Equivalence between Herding and Conditional Gradient Algorithms , 2012, ICML.

[37]  W. Rogosinski Moments of non-negative mass , 1958, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[38]  Bernhard Schölkopf,et al.  A New Distribution-Free Concept for Representing, Comparing, and Propagating Uncertainty in Dynamical Systems with Kernel Probabilistic Programming , 2019, ArXiv.