Topology-preserving flocking of nonlinear agents using optimistic planning

We consider the generalized flocking problem in multiagent systems, where the agents must drive a subset of their state variables to common values, while communication is constrained by a proximity relationship in terms of another subset of variables. We build a flocking method for general nonlinear agent dynamics, by using at each agent a near-optimal control technique from artificial intelligence called optimistic planning. By defining the rewards to be optimized in a well-chosen way, the preservation of the interconnection topology is guaranteed, under a controllability assumption. We also give a practical variant of the algorithm that does not require to know the details of this assumption, and show that it works well in experiments on nonlinear agents.

[1]  Jorge Cortes,et al.  Distributed Control of Robotic Networks: A Mathematical Approach to Motion Coordination Algorithms , 2009 .

[2]  George J. Pappas,et al.  Flocking in Fixed and Switching Networks , 2007, IEEE Transactions on Automatic Control.

[3]  Bart De Schutter,et al.  Approximate dynamic programming with a fuzzy parameterization , 2010, Autom..

[4]  George J. Pappas,et al.  Distributed connectivity control of mobile networks , 2007, 2007 46th IEEE Conference on Decision and Control.

[5]  Irinel-Constantin Morarescu,et al.  Convex Conditions on Decentralized Control for Graph Topology Preservation , 2014, IEEE Transactions on Automatic Control.

[6]  Frank L. Lewis,et al.  Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics , 2014, Autom..

[7]  Randal W. Beard,et al.  Distributed Consensus in Multi-vehicle Cooperative Control - Theory and Applications , 2007, Communications and Control Engineering.

[8]  Wenwu Yu,et al.  Flocking of multi-agent dynamical systems based on pseudo-leader mechanism , 2009, Syst. Control. Lett..

[9]  Panagiotis D. Christofides,et al.  Sequential and Iterative Architectures for Distributed Model Predictive Control of Nonlinear Process Systems , 2010 .

[10]  F.L. Lewis,et al.  Reinforcement learning and adaptive dynamic programming for feedback control , 2009, IEEE Circuits and Systems Magazine.

[11]  Bart De Schutter,et al.  Multi-agent model predictive control for transportation networks: Serial versus parallel schemes , 2008, Eng. Appl. Artif. Intell..

[12]  Luigi Fortuna,et al.  Reinforcement Learning and Adaptive Dynamic Programming for Feedback Control , 2009 .

[13]  Guangfu Ma,et al.  Distributed Coordinated Tracking With a Dynamic Leader for Multiple Euler-Lagrange Systems , 2011, IEEE Transactions on Automatic Control.

[14]  Frank L. Lewis,et al.  Reinforcement Learning and Approximate Dynamic Programming for Feedback Control , 2012 .

[15]  Rémi Munos,et al.  From Bandits to Monte-Carlo Tree Search: The Optimistic Principle Applied to Optimization and Planning , 2014, Found. Trends Mach. Learn..

[16]  Eduardo Sontag,et al.  Controllability of Nonlinear Discrete-Time Systems: A Lie-Algebraic Approach , 1990, SIAM Journal on Control and Optimization.

[17]  Claudio De Persis,et al.  Robust Self-Triggered Coordination With Ternary Controllers , 2012, IEEE Transactions on Automatic Control.

[18]  Guanrong Chen,et al.  Adaptive second-order consensus of networked mobile agents with nonlinear dynamics , 2011, Autom..

[19]  George J. Pappas,et al.  Flocking in Teams of Nonholonomic Agents , 2003 .

[20]  Xinghuo Yu,et al.  Flocking of Multi-Agent Non-Holonomic Systems With Proximity Graphs , 2013, IEEE Transactions on Circuits and Systems I: Regular Papers.

[21]  F. Knorn Topics in Cooperative Control , 2011 .

[22]  Lucian Busoniu,et al.  Consensus for black-box nonlinear agents using optimistic optimization , 2014, Autom..

[23]  Robert Babuska,et al.  A review of optimistic planning in Markov decision processes , 2013 .

[24]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[25]  Rémi Munos,et al.  Optimistic Planning of Deterministic Systems , 2008, EWRL.

[26]  Francesco Bullo,et al.  Distributed Control of Robotic Networks , 2009 .

[27]  Wenjie Dong,et al.  Flocking of Multiple Mobile Robots Based on Backstepping , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[28]  Reza Olfati-Saber,et al.  Consensus and Cooperation in Networked Multi-Agent Systems , 2007, Proceedings of the IEEE.

[29]  Reza Olfati-Saber,et al.  Flocking for multi-agent dynamic systems: algorithms and theory , 2006, IEEE Transactions on Automatic Control.

[30]  Lucian Buşoniu,et al.  Optimistic Planning for Consensus , 2013 .