Regularization for Design

When designing controllers for large-scale systems, the architectural aspects of the controller such as the placement of actuators, sensors, and the communication links between them can no longer be taken as given. The task of designing this architecture is now as important as the design of the control laws themselves. By interpreting controller synthesis (in a model matching setup) as the solution of a particular linear inverse problem, we view the challenge of obtaining a controller with a desired architecture as one of finding a structured solution to an inverse problem. Building on this conceptual connection, we formulate and analyze a framework called Regularization for Design (RFD), in which we augment the variational formulations of controller synthesis problems with convex penalty functions that induce a desired controller architecture. The resulting regularized formulations are convex optimization problems that can be solved efficiently; these convex programs provide a unified computationally tractable approach for the simultaneous co-design of a structured optimal controller and the actuation, sensing and communication architecture required to implement it. Further, these problems are natural control-theoretic analogs of prominent approaches such as the Lasso, the Group Lasso, the Elastic Net, and others that are employed in structured inference. In analogy to that literature, we show that our approach identifies optimally structured controllers under a suitable condition on a “signal-to-noise” type ratio.

[1]  Pablo A. Parrilo,et al.  $ {\cal H}_{2}$-Optimal Decentralized Control Over Posets: A State-Space Solution for State-Feedback , 2010, IEEE Transactions on Automatic Control.

[2]  M. Dahleh,et al.  Solution of MIMO H2/ℓ1 problem without zero interpolation , 1997 .

[3]  Pablo A. Parrilo,et al.  Optimal output feedback architecture for triangular LQG problems , 2014, 2014 American Control Conference.

[4]  Javad Lavaei,et al.  Convex relaxation for optimal distributed control problem , 2014, 53rd IEEE Conference on Decision and Control.

[5]  Nuno C. Martins,et al.  Youla-Like Parametrizations Subject to QI Subspace Constraints , 2014, IEEE Transactions on Automatic Control.

[6]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[7]  Fu Lin,et al.  On identifying sparse representations of consensus networks , 2012 .

[8]  Fu Lin,et al.  Sparse feedback synthesis via the alternating direction method of multipliers , 2012, 2012 American Control Conference (ACC).

[9]  Nikolai Matni,et al.  Localized LQG optimal control for large-scale systems , 2016, 2016 American Control Conference (ACC).

[10]  S. Lall,et al.  Quadratic invariance is necessary and sufficient for convexity , 2011, Proceedings of the 2011 American Control Conference.

[11]  D. Donoho For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[12]  Mi-Ching Tsai,et al.  Robust and Optimal Control , 2014 .

[13]  Bin Yu,et al.  High-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence , 2008, 0811.3628.

[14]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[15]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[16]  Takashi Tanaka,et al.  The Bounded Real Lemma for Internally Positive Systems and H-Infinity Structured Static State Feedback , 2011, IEEE Transactions on Automatic Control.

[17]  Anders Rantzer,et al.  Scalable control of positive systems , 2012, Eur. J. Control.

[18]  Nikolai Matni,et al.  Low-Rank and Low-Order Decompositions for Local System Identification , 2014, ArXiv.

[19]  Y. Ho,et al.  Team decision theory and information structures in optimal control problems--Part II , 1972 .

[20]  Jianqing Fan,et al.  Sparsistency and Rates of Convergence in Large Covariance Matrix Estimation. , 2007, Annals of statistics.

[21]  Javad Lavaei,et al.  Convex Relaxation for Optimal Distributed Control Problems , 2014, IEEE Transactions on Automatic Control.

[22]  Soummya Kar,et al.  Minimum cost constrained input-output and control configuration co-design problem: A structural systems approach , 2015, 2015 American Control Conference (ACC).

[23]  Stephen Smale,et al.  Finding the Homology of Submanifolds with High Confidence from Random Samples , 2008, Discret. Comput. Geom..

[24]  L. Jones A Simple Lemma on Greedy Approximation in Hilbert Space and Convergence Rates for Projection Pursuit Regression and Neural Network Training , 1992 .

[25]  Stephen P. Boyd,et al.  Distributed average consensus with least-mean-square deviation , 2007, J. Parallel Distributed Comput..

[26]  Evangelos Theodorou,et al.  Convexity of optimal linear controller design , 2013, 52nd IEEE Conference on Decision and Control.

[27]  Sanjay Lall,et al.  A Characterization of Convex Problems in Decentralized Control$^ast$ , 2005, IEEE Transactions on Automatic Control.

[28]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[29]  Richard M. Murray,et al.  A scalable formulation for engineering combination therapies for evolutionary dynamics of disease , 2013, 2014 American Control Conference.

[30]  J. Doyle,et al.  Robust and optimal control , 1995, Proceedings of 35th IEEE Conference on Decision and Control.

[31]  Stephen P. Boyd,et al.  A rank minimization heuristic with application to minimum order system approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[32]  Emanuel Todorov,et al.  Convex Structured Controller Design , 2013, ArXiv.

[33]  Maxim Kristalny,et al.  On structured realizability and stabilizability of linear systems , 2013, 2013 American Control Conference.

[34]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[35]  Venkat Chandrasekaran,et al.  Regularization for Design , 2014, IEEE Transactions on Automatic Control.

[36]  M. Fardad,et al.  Sparsity-promoting optimal control for a class of distributed systems , 2011, Proceedings of the 2011 American Control Conference.

[37]  Nikolai Matni,et al.  A convex approach to sparse ℋ∞ analysis & synthesis , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[38]  Fu Lin,et al.  Design of Optimal Sparse Interconnection Graphs for Synchronization of Oscillator Networks , 2013, IEEE Transactions on Automatic Control.

[39]  Nuno C. Martins,et al.  Stabilizability and Norm-Optimal Control Design subject to Sparsity Constraints , 2012, ArXiv.

[40]  Nuno C. Martins,et al.  Information structures in optimal decentralized control , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[41]  Murti V. Salapaka,et al.  Architectures for Distributed Controller With Sub-Controller Communication Uncertainty , 2010, IEEE Transactions on Automatic Control.

[42]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[43]  Laurent Lessard,et al.  Optimal decentralized state-feedback control with sparsity and delays , 2013, Autom..

[44]  F. Bonsall A GENERAL ATOMIC DECOMPOSITION THEOREM AND BANACH'S CLOSED RANGE THEOREM , 1991 .

[45]  Fu Lin,et al.  Design of Optimal Sparse Feedback Gains via the Alternating Direction Method of Multipliers , 2011, IEEE Transactions on Automatic Control.

[46]  H. Witsenhausen A Counterexample in Stochastic Optimum Control , 1968 .

[47]  Nikolai Matni Distributed control subject to delays satisfying an ℋ1 norm bound , 2014, 53rd IEEE Conference on Decision and Control.

[48]  Jean-Philippe Vert,et al.  Group Lasso with Overlaps: the Latent Group Lasso approach , 2011, ArXiv.

[49]  Nikolai Matni,et al.  Localized distributed state feedback control with communication delays , 2014, 2014 American Control Conference.

[50]  Lennart Ljung,et al.  Some Classical and Some New Ideas for Identification of Linear Systems , 2013 .

[51]  Nikolai Matni,et al.  Localized distributed optimal control with output feedback and communication delays , 2014, 2014 52nd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[52]  John Doyle,et al.  Output feedback ℌ2 model matching for decentralized systems with delays , 2012, 2013 American Control Conference.

[53]  Nikolai Matni Communication delay co-design in ℌ2 decentralized control Using atomic norm minimization , 2013, 52nd IEEE Conference on Decision and Control.

[54]  John Doyle,et al.  The $H_2$ Control Problem for Decentralized Systems with Delays , 2013, ArXiv.

[55]  Nikolai Matni,et al.  A Convex Approach to Sparse H infinity Analysis & Synthesis , 2015, ArXiv.

[56]  M. Wainwright Structured Regularizers for High-Dimensional Problems: Statistical and Computational Issues , 2014 .

[57]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[58]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[59]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[60]  Javad Lavaei,et al.  Efficient convex relaxation for stochastic optimal distributed control problem , 2014, 2014 52nd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[61]  Sanjay Lall,et al.  A state-space solution to the two-player decentralized optimal control problem , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[62]  Martin J. Wainwright,et al.  Restricted Eigenvalue Properties for Correlated Gaussian Designs , 2010, J. Mach. Learn. Res..

[63]  Nader Motee,et al.  Sparsity measures for spatially decaying systems , 2014, 2014 American Control Conference.

[64]  Carsten W. Scherer,et al.  Structured H∞-optimal control for nested interconnections: A state-space solution , 2013, Syst. Control. Lett..

[65]  Randy Cogill,et al.  Convexity of optimal control over networks with delays and arbitrary topology , 2010 .

[66]  Andrew G. Lamperski,et al.  Optimal state-feedback control under sparsity and delay constraints , 2012 .

[67]  Nikolai Matni Communication Delay Co-Design in $\mathcal{ H}_{2}$-Distributed Control Using Atomic Norm Minimization , 2017, IEEE Transactions on Control of Network Systems.

[68]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[69]  M. Dahleh,et al.  Control of Uncertain Systems: A Linear Programming Approach , 1995 .

[70]  Pablo A. Parrilo,et al.  ℋ2-optimal decentralized control over posets: A state space solution for state-feedback , 2010, 49th IEEE Conference on Decision and Control (CDC).

[71]  Parikshit Shah,et al.  Linear system identification via atomic norm regularization , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[72]  Ronald A. DeVore,et al.  Some remarks on greedy algorithms , 1996, Adv. Comput. Math..

[73]  Zhi-Quan Luo,et al.  An ADMM algorithm for optimal sensor and actuator selection , 2014, 53rd IEEE Conference on Decision and Control.

[74]  John C. Doyle,et al.  The H2 Control Problem for Quadratically Invariant Systems with Delays , 2013 .

[75]  G. Pisier Probabilistic methods in the geometry of Banach spaces , 1986 .

[76]  Jean-Philippe Vert,et al.  Group lasso with overlap and graph lasso , 2009, ICML '09.

[77]  A. E. Hoerl,et al.  Ridge regression: biased estimation for nonorthogonal problems , 2000 .

[78]  S. Geer,et al.  On the conditions used to prove oracle results for the Lasso , 2009, 0910.0722.

[79]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[80]  Nikolai Matni,et al.  Localized LQR optimal control , 2014, 53rd IEEE Conference on Decision and Control.