A statistical learning theory approach for uncertain linear and bilinear matrix inequalities

Abstract In this paper, we consider the problem of minimizing a linear functional subject to uncertain linear and bilinear matrix inequalities, which depend in a possibly nonlinear way on a vector of uncertain parameters. Motivated by recent results in statistical learning theory, we show that probabilistic guaranteed solutions can be obtained by means of randomized algorithms. In particular, we show that the Vapnik–Chervonenkis dimension (VC-dimension) of the two problems is finite, and we compute upper bounds on it. In turn, these bounds allow us to derive explicitly the sample complexity of these problems. Using these bounds, in the second part of the paper, we derive a sequential scheme, based on a sequence of optimization and validation steps. The algorithm is on the same lines of recent schemes proposed for similar problems, but improves both in terms of complexity and generality. The effectiveness of this approach is shown using a linear model of a robot manipulator subject to uncertain parameters.

[1]  Giuseppe Carlo Calafiore,et al.  A probabilistic analytic center cutting plane method for feasibility of uncertain LMIs , 2007, Autom..

[2]  R. Braatz,et al.  A tutorial on linear and bilinear matrix inequalities , 2000 .

[3]  Maryamsadat Tahavori,et al.  IEEE Multi-Conference on Systems and Control , 2013 .

[4]  Roberto Tempo,et al.  Probabilistic robust design with linear quadratic regulators , 2001, Syst. Control. Lett..

[5]  Giuseppe Carlo Calafiore,et al.  Research on probabilistic methods for control system design , 2011, Autom..

[6]  Mathukumalli Vidyasagar,et al.  Randomized algorithms for robust controller synthesis using statistical learning theory , 2001, Autom..

[7]  E. Yaz.  Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.

[8]  Mathukumalli Vidyasagar,et al.  Learning and Generalization: With Applications to Neural Networks , 2002 .

[9]  R. Tempo,et al.  Randomized Algorithms for Analysis and Control of Uncertain Systems , 2004 .

[10]  Mario Sznaier,et al.  Randomized Algorithms for Analysis and Control of Uncertain Systems with Applications, Second Edition, Roberto Tempo, Giuseppe Calafiore, Fabrizio Dabbene (Eds.). Springer-Verlag, London (2013), 357, ISBN: 978-1-4471-4609-4 , 2014, Autom..

[11]  Michel Verhaegen,et al.  Robust output-feedback controller design via local BMI optimization , 2004, Autom..

[12]  Michel Verhaegen,et al.  An ellipsoid algorithm for probabilistic robust controller design , 2003, Syst. Control. Lett..

[13]  Laurent El Ghaoui,et al.  Robust Solutions to Uncertain Semidefinite Programs , 1998, SIAM J. Optim..

[14]  Giuseppe Carlo Calafiore,et al.  Stochastic algorithms for exact and approximate feasibility of robust LMIs , 2001, IEEE Trans. Autom. Control..

[15]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[16]  Eduardo F. Camacho,et al.  Randomized Strategies for Probabilistic Solutions of Uncertain Feasibility and Optimization Problems , 2009, IEEE Transactions on Automatic Control.

[17]  Yasuaki Oishi,et al.  Polynomial-time algorithms for probabilistic solutions of parameter-dependent linear matrix inequalities , 2007, Autom..

[18]  Giuseppe Carlo Calafiore,et al.  The scenario approach to robust control design , 2006, IEEE Transactions on Automatic Control.

[19]  Marek Karpinski,et al.  Polynomial Bounds for VC Dimension of Sigmoidal and General Pfaffian Neural Networks , 1997, J. Comput. Syst. Sci..

[20]  Vincent D. Blondel,et al.  Probabilistic solutions to some NP-hard matrix problems , 2001, Autom..

[21]  Johan Efberg,et al.  YALMIP : A toolbox for modeling and optimization in MATLAB , 2004 .

[22]  Marco C. Campi,et al.  The Exact Feasibility of Randomized Solutions of Uncertain Convex Programs , 2008, SIAM J. Optim..

[23]  F. Leibfritz.  COMPleib: COnstrained Matrix–optimization Problem library – a collection of test examples for nonlinear semidefinite programs, control system design and related problems , 2006 .

[24]  Dmitry Panchenko,et al.  Improved sample complexity estimates for statistical learning control of uncertain systems , 2000, IEEE Trans. Autom. Control..

[25]  Roberto Tempo,et al.  Randomized methods for design of uncertain systems: Sample complexity and sequential algorithms , 2013, Autom..

[26]  Jun'ichi Takeuchi.  Improved Sample Complexity Bounds for Parameter Estimation , 1995, IEICE Trans. Inf. Syst..

[27]  Qing-Guo Wang,et al.  Sequential Randomized Algorithms for Convex Optimization in the Presence of Uncertainty , 2013, IEEE Transactions on Automatic Control.

[28]  Michael G. Safonov,et al.  Global optimization for the Biaffine Matrix Inequality problem , 1995, J. Glob. Optim..

[29]  D. Bernstein.  Matrix Mathematics: Theory, Facts, and Formulas , 2009 .

[30]  Qing-Guo Wang,et al.  On the sample complexity of uncertain linear and bilinear matrix inequalities , 2013, 52nd IEEE Conference on Decision and Control.

[31]  Qing-Guo Wang,et al.  Sequential randomized algorithms for sampled convex optimization , 2013, 2013 IEEE Conference on Computer Aided Control System Design (CACSD).

[32]  G. Calafiore,et al.  Reduced Vertex Set Result for Interval Semidefinite Optimization Problems , 2008 .

[33]  Giuseppe Carlo Calafiore,et al.  RACT: Randomized Algorithms Control Toolbox for MATLAB , 2008 .

[34]  Stephen P. Boyd,et al.  Linear Matrix Inequalities in Systems and Control Theory , 1994 .

[35]  Eduardo F. Camacho,et al.  A new vertex result for robustness problems with interval matrix uncertainty , 2007, 2007 European Control Conference (ECC).

[36]  S. Uryasev.  Probabilistic constrained optimization : methodology and applications , 2000 .

[37]  Michel Verhaegen,et al.  Controller reconfiguration for non-linear systems , 2000 .

[38]  D. Bernstein.  Matrix Mathematics: Theory, Facts, and Formulas , 2009 .

[39]  Norbert Sauer,et al.  On the Density of Families of Sets , 1972, J. Comb. Theory, Ser. A.

[40]  Giuseppe Carlo Calafiore,et al.  Uncertain convex programs: randomized solutions and confidence levels , 2005, Math. Program..

[41]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[42]  Stephen P. Boyd,et al.  Semidefinite Programming , 1996, SIAM Rev..

[43]  Roberto Tempo,et al.  The Sample Complexity of Randomized Methods for Analysis and Design of Uncertain Systems , 2013, ArXiv.

[44]  Y. Fujisaki,et al.  Probabilistic robust controller design: probable near minimax value and randomized algorithms , 2003 .

[45]  Johan Löfberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004 .

[46]  Michael Stingl,et al.  PENNON: A code for convex nonlinear and semidefinite programming , 2003, Optim. Methods Softw..

[47]  Roberto Tempo,et al.  Randomized control design through probabilistic validation , 2012, 2012 American Control Conference (ACC).