Gradient Methods for Large-Scale and Distributed Linear Quadratic Control

This thesis considers methods for synthesis of linear quadratic controllers for large-scale, interconnected systems. Conventional methods that solve the linear quadratic control problem are only applicable to systems with moderate size, due to the rapid increase in both computational time and memory requirements as the system size increases. The methods presented in this thesis show a much slower increase in these requirements when faced with system matrices with a sparse structure. Hence, they are useful for control design for systems of large order, since they usually have sparse systems matrices. An equally important feature of the methods is that the controllers are restricted to have a distributed nature, meaning that they respect a potential interconnection structure of the system. The controllers considered in the thesis have the same structure as the centralized LQG solution, that is, they are consisting of a state predictor and feedback from the estimated states. Strategies for determining the feedback matrix and predictor matrix separately, are suggested. The strategies use gradient directions of the cost function to iteratively approach a locally optimal solution in either problem. A scheme to determine bounds on the degree of suboptimality of the partial solution in every iteration, is presented. It is also shown that these bounds can be combined to give a bound on the degree of suboptimality of the full output feedback controller. Another method that treats the synthesis of the feedback matrix and predictor matrix simultaneously is also presented. The functionality of the developed methods is illustrated by an application, where the methods are used to compute controllers for a large deformable mirror, found in a telescope to compensate for atmospheric disturbances. The model of the mirror is obtained by discretizing a partial differential equation. This gives a linear, sparse representation of the mirror with a very large state space, which is suitable for the methods presented in the thesis. The performance of the controllers is evaluated using performance measures from the adaptive optics community.

[1]  O. Staffans On the discrete and continuous time infinite-dimensional algebraic Riccati equations , 1996 .

[2]  Kanat Camlibel,et al.  Proc. of the 46th IEEE Conference on Decision and Control , 2007 .

[3]  Valeria Simoncini,et al.  A New Iterative Method for Solving Large-Scale Lyapunov Matrix Equations , 2007, SIAM J. Sci. Comput..

[4]  Michael Rotkowitz On information structures, convexity, and linear optimality , 2008, 2008 47th IEEE Conference on Decision and Control.

[5]  R. Noll Zernike polynomials and atmospheric turbulence , 1976 .

[6]  Geir E. Dullerud,et al.  Distributed control of heterogeneous systems , 2004, IEEE Transactions on Automatic Control.

[7]  Y. Zhou,et al.  On the decay rate of Hankel singular values and related issues , 2002, Syst. Control. Lett..

[8]  Ywh-Pyng Ham Optimal Low-Order Controller Design via LQG-Like Parametrization * , 1992 .

[9]  A. Rantzer,et al.  A Separation Principle for Distributed Control , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[10]  Alan J. Laub,et al.  Matrix sign function implementations on a hypercube multiprocessor , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[11]  Torben Andersen,et al.  Integrated Modeling of Telescopes , 2011 .

[12]  Giorgio Battistelli,et al.  Model-free Adaptive Switching Control of Uncertain Time-Varying Plants , 2011 .

[13]  Danny C. Sorensen,et al.  A Modified Low-Rank Smith Method for Large-Scale Lyapunov Equations , 2004, Numerical Algorithms.

[14]  Keiichiro Yasuda,et al.  Optimality of decentralized control for large-scale systems , 1983, Autom..

[15]  Raffaello D'Andrea,et al.  A decomposition approach to multi-vehicle cooperative control , 2005, Robotics Auton. Syst..

[16]  Petros G. Voulgaris,et al.  A convex characterization of distributed control problems in spatially invariant systems with communication constraints , 2005, Syst. Control. Lett..

[17]  Jean-Marie Buchot,et al.  An invariant subspace method for large-scale algebraic Riccati equation , 2010 .

[18]  M. Lazar,et al.  Distributed predictive control of the 7-machine CIGRÉ power system , 2011, Proceedings of the 2011 American Control Conference.

[19]  Gene H. Golub,et al.  Matrix computations , 1983 .

[20]  Cédric Langbort,et al.  Distributed control design for systems interconnected over an arbitrary graph , 2004, IEEE Transactions on Automatic Control.

[21]  Boris N. Khoromskij,et al.  Solution of Large Scale Algebraic Matrix Riccati Equations by Use of Hierarchical Matrices , 2003, Computing.

[22]  Francois Roddier,et al.  Adaptive Optics in Astronomy: Imaging through the atmosphere , 2004 .

[23]  David Q. Mayne,et al.  Constrained model predictive control: Stability and optimality , 2000, Autom..

[24]  J. Marschak,et al.  Elements for a Theory of Teams , 1955 .

[25]  Uri M. Ascher,et al.  The Chaotic Nature of Faster Gradient Descent Methods , 2012, J. Sci. Comput..

[26]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[27]  P. P. J. van den Bosch,et al.  Distributed Price-based Optimal Control of Power Systems , 2007, 2007 IEEE International Conference on Control Applications.

[28]  John Langford,et al.  Sparse Online Learning via Truncated Gradient , 2008, NIPS.

[29]  H. Witsenhausen A Counterexample in Stochastic Optimum Control , 1968 .

[30]  Torben Andersen,et al.  Novel concept for large deformable mirrors , 2006 .

[31]  Y. Ho,et al.  Team decision theory and information structures in optimal control problems--Part II , 1972 .

[32]  Romeo Ortega,et al.  Passivity of Nonlinear Incremental Systems: Application to PI Stabilization of Nonlinear RLC Circuits , 2006, CDC.

[33]  Stephen P. Boyd,et al.  Linear controller design: limits of performance , 1991 .

[35]  J. Reddy An introduction to the finite element method , 1989 .

[36]  Rikard Heimsten,et al.  Study of a Large Deformable Mirror Concept , 2011 .

[37]  Wei Xu,et al.  Towards Optimal One Pass Large Scale Learning with Averaged Stochastic Gradient Descent , 2011, ArXiv.

[38]  Geir E. Dullerud,et al.  Distributed control design for spatially interconnected systems , 2003, IEEE Trans. Autom. Control..

[39]  Sanjay Lall,et al.  A Characterization of Convex Problems in Decentralized Control$^ast$ , 2005, IEEE Transactions on Automatic Control.

[40]  K. Åström Introduction to Stochastic Control Theory , 1970 .

[41]  W. Hager,et al.  The cyclic Barzilai-–Borwein method for unconstrained optimization , 2006 .

[42]  Arthur E. Bryson,et al.  Applied Optimal Control , 1969 .

[43]  Armando Riccardi,et al.  First light AO (FLAO) system for LBT: performance analysis and optimization , 2010, Astronomical Telescopes + Instrumentation.

[44]  Li Li,et al.  LMI relaxation to Riccati equations in structured H2 control , 2006, 2006 American Control Conference.

[45]  Michel Verhaegen,et al.  A Decomposition Approach to Distributed Control of Dynamic Deformable Mirrors , 2010 .

[46]  Francesco Borrelli,et al.  Distributed LQR Design for Identical Dynamically Decoupled Systems , 2008, IEEE Transactions on Automatic Control.

[47]  Leon S. Lasdon,et al.  Optimization Theory of Large Systems , 1970 .

[48]  Joe H. Chow,et al.  A Feedback Descent Method for Solving Constrained LQG Control Problems , 1992, 1992 American Control Conference.

[49]  A. Barraud A numerical algorithm to solve AT X A - X = Q , 1977, 1977 IEEE Conference on Decision and Control including the 16th Symposium on Adaptive Processes and A Special Symposium on Fuzzy Set Theory and Applications.

[50]  Mihailo R. Jovanovic,et al.  On the peaking phenomenon in the control of vehicular platoons , 2008, Syst. Control. Lett..

[51]  E. Denman,et al.  The matrix sign function and computations in systems , 1976 .

[52]  M. Mesarovic,et al.  Theory of Hierarchical, Multilevel, Systems , 1970 .

[53]  José Claudio Geromel,et al.  Optimal decentralized control of dynamic systems , 1982, Autom..

[54]  S. Lall,et al.  Optimal controller synthesis for a decentralized two-player system with partial output feedback , 2011, Proceedings of the 2011 American Control Conference.

[55]  R. Krtolica,et al.  A stochastic inclusion principle , 1983, The 22nd IEEE Conference on Decision and Control.

[56]  A. Gattami Generalized Linear Quadratic Control Theory , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[57]  Peter Benner,et al.  Numerical solution of large‐scale Lyapunov equations, Riccati equations, and linear‐quadratic optimal control problems , 2008, Numer. Linear Algebra Appl..

[58]  S. Lall,et al.  Decentralized control information structures preserved under feedback , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[59]  Richard M. Murray,et al.  DISTRIBUTED SENSOR FUSION USING DYNAMIC CONSENSUS , 2005 .

[60]  Jean-Marc Conan,et al.  Linear quadratic Gaussian control for adaptive optics and multiconjugate adaptive optics: experimental and numerical analysis. , 2009, Journal of the Optical Society of America. A, Optics, image science, and vision.

[61]  C. Kulcsár,et al.  Optimal control law for classical and multiconjugate adaptive optics. , 2004, Journal of the Optical Society of America. A, Optics, image science, and vision.

[62]  C. Knapp,et al.  Parameter optimization in linear systems with arbitrarily constrained controller structure , 1979, 1979 18th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[63]  C. Kulcsár,et al.  Optimal control for Multi-Conjugate Adaptive Optics , 2005 .

[64]  Khalide Jbilou,et al.  Block Krylov Subspace Methods for Large Algebraic Riccati Equations , 2003, Numerical Algorithms.

[65]  Roger Fletcher,et al.  On the asymptotic behaviour of some new gradient methods , 2005, Math. Program..

[66]  Fernando Paganini,et al.  Distributed control of spatially invariant systems , 2002, IEEE Trans. Autom. Control..

[67]  Armando Riccardi,et al.  First light of the 6.5-m MMT adaptive optics system , 2003, SPIE Optics + Photonics.

[68]  M. Athans,et al.  Solution of some nonclassical LQG stochastic decision problems , 1974 .

[69]  D. D. iljak,et al.  Paper: Estimation and control of large sparse systems , 1985 .

[70]  William B. Dunbar,et al.  Distributed receding horizon control for multi-vehicle formation stabilization , 2006, Autom..

[71]  Kyle A. Gallivan,et al.  Singular Riccati equations stabilizing large-scale systems , 2006 .

[72]  Peter Benner,et al.  On the numerical solution of large-scale sparse discrete-time Riccati equations , 2011, Adv. Comput. Math..

[73]  David W. Miller,et al.  Robust control of the Multiple Mirror Telescope adaptive secondary mirror , 1999 .

[74]  Thilo Penzl,et al.  A Cyclic Low-Rank Smith Method for Large Sparse Lyapunov Equations , 1998, SIAM J. Sci. Comput..

[75]  Francesco Borrelli,et al.  Decentralized receding horizon control for large scale dynamically decoupled systems , 2009, Autom..

[76]  Dragoslav D. Siljak,et al.  Estimation and control of large sparse systems , 1985, Autom..

[77]  Bart De Schutter,et al.  Multi-agent model predictive control for transportation networks: Serial versus parallel schemes , 2008, Eng. Appl. Artif. Intell..

[78]  R. Paschall,et al.  Linear quadratic Gaussian control of a deformable mirror adaptive optics system with time-delayed measurements. , 1993, Applied optics.

[79]  Michael Athans,et al.  Survey of decentralized control methods for large scale systems , 1978 .

[80]  G. Hewer An iterative technique for the computation of the steady state gains for the discrete optimal regulator , 1971 .

[81]  Léon Bottou,et al.  Stochastic Learning , 2003, Advanced Lectures on Machine Learning.

[82]  Ruth F. Curtain,et al.  Proceedings of the 48th IEEE Conference on Decision and Control, CDC 2009, combined withe the 28th Chinese Control Conference, December 16-18, 2009, Shanghai, China , 2009, CDC.

[83]  J. Hardy,et al.  Adaptive Optics for Astronomical Telescopes , 1998 .

[84]  Hugh F. Durrant-Whyte,et al.  A Fully Decentralized Multi-Sensor System For Tracking and Surveillance , 1993, Int. J. Robotics Res..

[85]  Riccardo Scattolini,et al.  Architectures for distributed and hierarchical Model Predictive Control - A review , 2009 .

[86]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[87]  K. Jbilou,et al.  Projection methods for large Lyapunov matrix equations , 2006 .

[88]  A. Rantzer,et al.  A Scalable Modularized Synthesis Method for Distributed Kalman Filters , 2011 .

[89]  Reza Olfati-Saber,et al.  Distributed Kalman filtering for sensor networks , 2007, 2007 46th IEEE Conference on Decision and Control.

[90]  Judith Gardiner,et al.  A generalization of the matrix sign function solution for algebraic Riccati equations , 1985, 1985 24th IEEE Conference on Decision and Control.

[91]  Thilo Penzl Eigenvalue decay bounds for solutions of Lyapunov equations: the symmetric case , 2000 .

[92]  R. Radner,et al.  Team Decision Problems , 1962 .

[93]  Y. Saad,et al.  Numerical solution of large Lyapunov equations , 1989 .

[94]  S. Stankovic Stochastic inclusion principle applied to decentralized automatic generation control , 1999 .

[95]  Anders Rantzer,et al.  Sub-Optimality Bound on a Gradient Method for Iterative Distributed Control Synthesis , 2010 .

[96]  I. Jaimoukha,et al.  Krylov subspace methods for solving large Lyapunov equations , 1994 .

[97]  Jacob K. White,et al.  Low Rank Solution of Lyapunov Equations , 2002, SIAM J. Matrix Anal. Appl..

[98]  H. Bock,et al.  A Multiple Shooting Algorithm for Direct Solution of Optimal Control Problems , 1984 .

[99]  Rodney Teo,et al.  Decentralized overlapping control of a formation of unmanned aerial vehicles , 2004, Autom..

[100]  A. Laub A schur method for solving algebraic Riccati equations , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[101]  Wolfgang Dahmen,et al.  Introduction to Model Based Optimization of Chemical Processes on Moving Horizons , 2001 .

[102]  Michel Verhaegen,et al.  Exploiting the spatiotemporal correlation in adaptive optics using data-driven H2-optimal control. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[103]  E. Davison,et al.  On the stabilization of decentralized control systems , 1973 .

[104]  Rafael Castro-Linares,et al.  Trajectory tracking for non-holonomic cars: A linear approach to controlled leader-follower formation , 2010, 49th IEEE Conference on Decision and Control (CDC).

[105]  Irena Lasiecka,et al.  Riccati theory and singular estimates for a Bolza control problem arising in linearized fluid-structure interaction , 2009, Syst. Control. Lett..

[106]  Fu Lin,et al.  Augmented Lagrangian Approach to Design of Structured Optimal State Feedback Gains , 2011, IEEE Transactions on Automatic Control.

[107]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[108]  P. Benner,et al.  Initializing Newton's method for discrete-time algebraic Riccati equations using the butterfly SZ algorithm , 1999, Proceedings of the 1999 IEEE International Symposium on Computer Aided Control System Design (Cat. No.99TH8404).

[109]  Lars Grasedyck,et al.  Nonlinear multigrid for the solution of large‐scale Riccati equations in low‐rank and ℋ︁‐matrix format , 2008, Numer. Linear Algebra Appl..

[110]  E. Todorov,et al.  A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems , 2005, Proceedings of the 2005, American Control Conference, 2005..

[111]  Bruno Iannazzo,et al.  Numerical Solution of Algebraic Riccati Equations , 2012, Fundamentals of algorithms.

[112]  Douglas P Looze Linear-quadratic-Gaussian control for adaptive optics systems using a hybrid model. , 2009, Journal of the Optical Society of America. A, Optics, image science, and vision.

[113]  Byron M. Welsh,et al.  Modal control for an adaptive optics system using LQG compensation , 1992 .

[114]  J. Doyle,et al.  Robust and optimal control , 1995, Proceedings of 35th IEEE Conference on Decision and Control.

[115]  Tae Woong Yoon,et al.  Proceedings of the 43rd IEEE Conference on Decision and Control , 2004 .