Graph Neural Networks for Decentralized Controllers

Dynamical systems comprised of autonomous agents arise in many relevant problems such as multi-agent robotics, smart grids, or smart cities. Controlling these systems is of paramount importance to guarantee a successful deployment. Optimal centralized controllers are readily available but face limitations in terms of scalability and practical implementation. Optimal decentralized controllers, on the other hand, are difficult to find. In this paper, we propose a framework using graph neural networks (GNNs) to learn decentralized controllers from data. While GNNs are naturally distributed architectures, making them perfectly suited for the task, we adapt them to handle delayed communications as well. Furthermore, they are equivariant and stable, leading to good scalability and transferability properties. The problem of flocking is explored to illustrate the potential of GNNs in learning decentralized controllers.

[1]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[2]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[3]  Antonio G. Marques,et al.  Convolutional Graph Neural Networks , 2019, 2019 53rd Asilomar Conference on Signals, Systems, and Computers.

[4]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[5]  Vijay Kumar,et al.  Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks , 2019, CoRL.

[6]  Graphs, Convolutions, and Neural Networks , 2020, ArXiv.

[7]  Naresh K. Sinha,et al.  Modern Control Systems , 1981, IEEE Transactions on Systems, Man, and Cybernetics.

[8]  Yonina C. Eldar,et al.  A unified view of diffusion maps and signal processing on graphs , 2017, 2017 International Conference on Sampling Theory and Applications (SampTA).

[9]  Fernando Gama,et al.  Stability Properties of Graph Neural Networks , 2019, IEEE Transactions on Signal Processing.

[10]  Yann LeCun,et al.  Spectral Networks and Deep Locally Connected Networks on Graphs , 2014 .

[11]  Cyrus Shahabi,et al.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ICLR.

[12]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[13]  H. Witsenhausen.  A Counterexample in Stochastic Optimum Control , 1968 .

[14]  J. Andrew Bagnell,et al.  Efficient Reductions for Imitation Learning , 2010, AISTATS.

[15]  Fernando Gama,et al.  Stability of Graph Scattering Transforms , 2019, NeurIPS.

[16]  Daniel Pérez Palomar,et al.  Power Control By Geometric Programming , 2007, IEEE Transactions on Wireless Communications.

[17]  Alejandro Ribeiro,et al.  Controllability of Bandlimited Graph Processes Over Random Time Varying Graphs , 2019, IEEE Transactions on Signal Processing.

[18]  Hamed Mohsenian Rad,et al.  Optimal Residential Load Control With Price Prediction in Real-Time Electricity Pricing Environments , 2010, IEEE Transactions on Smart Grid.

[19]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs , 2012, IEEE Transactions on Signal Processing.

[20]  Asuman E. Ozdaglar,et al.  Constrained Consensus and Optimization in Multi-Agent Networks , 2008, IEEE Transactions on Automatic Control.

[21]  Pierre Vandergheynst,et al.  Graph Signal Processing: Overview, Challenges, and Applications , 2017, Proceedings of the IEEE.

[22]  Herbert G. Tanner.  Flocking with obstacle avoidance in switching networks of interconnected vehicles , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[23]  Fernando Gama,et al.  From Graph Filters to Graph Neural Networks , 2020 .

[24]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[25]  Fernando Gama,et al.  Optimal Power Flow Using Graph Neural Networks , 2019, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[26]  Antonio G. Marques,et al.  Aggregation Graph Neural Networks , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[27]  Fernando Gama,et al.  EdgeNets: Edge Varying Graph Neural Networks , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Eduardo D. Sontag,et al.  Mathematical Control Theory: Deterministic Finite Dimensional Systems , 1990 .

[29]  Fernando Gama,et al.  Graph Neural Networks for Decentralized Multi-Robot Path Planning , 2019, ArXiv.

[30]  George J. Pappas,et al.  Stable flocking of mobile agents part I: dynamic topology , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[31]  Antonio G. Marques,et al.  Convolutional Neural Network Architectures for Signals Supported on Graphs , 2018, IEEE Transactions on Signal Processing.