Deep reinforcement learning of event-triggered communication and control for multi-agent cooperative transport

In this paper, we explore a multi-agent reinforcement learning approach to address the design problem of communication and control strategies for multi-agent cooperative transport. Typical end-to-end deep neural network policies may be insufficient for covering communication and control; these methods cannot decide the timing of communication and can only work with fixed-rate communications. Therefore, our framework exploits event-triggered architecture, namely, a feedback controller that computes the communication input and a triggering mechanism that determines when the input has to be updated again. Such event-triggered control policies are efficiently optimized using a multi-agent deep deterministic policy gradient. We confirmed that our approach could balance the transport performance and communication savings through numerical simulations.

[1]  Yi Wu,et al.  Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments , 2017, NIPS.

[2]  Vincent Berenz,et al.  Learning Event-triggered Control from Data through Joint Optimization , 2020, ArXiv.

[3]  Vijay Kumar,et al.  Decentralized Algorithm for Force Distribution With Applications to Cooperative Transport , 2015 .

[4]  Vijay Kumar,et al.  Dynamics, Control and Planning for Cooperative Manipulation of Payloads Suspended by Cables from Multiple Quadrotor Robots , 2013, Robotics: Science and Systems.

[5]  Sandra Hirche,et al.  Distributed Control for Cooperative Manipulation With Event-Triggered Communication , 2020, IEEE Transactions on Robotics.

[6]  Jens Kober,et al.  Human-Robot Cooperative Object Manipulation with Contact Changes , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[7]  Vijay Kumar,et al.  Composition of Vector Fields for Multi-Robot Manipulation via Caging , 2007, Robotics: Science and Systems.

[8]  Sandra Hirche,et al.  Multi-robot manipulation controlled by a human with haptic feedback , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[9]  Mac Schwager,et al.  Kinematic multi-robot manipulation with no communication using force feedback , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Mac Schwager,et al.  Force-Amplifying N-robot Transport System (Force-ANTS) for cooperative planar manipulation without communication , 2016, Int. J. Robotics Res..

[11]  Guang Yang,et al.  OuijaBots: Omnidirectional Robots for Cooperative Object Transport with Rotation Control Using No Communication , 2016, DARS.

[12]  Vijay Kumar,et al.  Cooperative Grasping and Transport Using Multiple Quadrotors , 2010, DARS.

[13]  Vijay Kumar,et al.  Cooperative manipulation and transportation with aerial robots , 2009, Auton. Robots.

[14]  Jonathan P. How,et al.  R-MADDPG for Partially Observable Environments and Limited Communication , 2019, ArXiv.

[15]  Antonio Franchi,et al.  Decentralized parameter estimation and observation for cooperative mobile manipulation of an unknown load using noisy measurements , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Mac Schwager,et al.  Multi-robot manipulation with no communication using only local measurements , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[17]  Wojciech Zaremba,et al.  Domain randomization for transferring deep neural networks from simulation to the real world , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[18]  Sebastian Trimpe,et al.  Deep Reinforcement Learning for Event-Triggered Control , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[19]  Marek Miskowicz,et al.  Event-Based Control and Signal Processing , 2015 .

[20]  Mac Schwager,et al.  Decentralized Adaptive Control for Collaborative Manipulation , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[21]  Vijay Kumar,et al.  The Inverse Kinematics of Cooperative Transport With Multiple Aerial Robots , 2013, IEEE Transactions on Robotics.

[22]  M. Ani Hsieh,et al.  Multi-robot manipulation via caging in environments with obstacles , 2008, 2008 IEEE International Conference on Robotics and Automation.

[23]  Antonio Franchi,et al.  Distributed estimation of the inertial parameters of an unknown load via multi-robot manipulation , 2014, 53rd IEEE Conference on Decision and Control.

[24]  Vijay Kumar,et al.  Cooperative Transportation Using Small Quadrotors Using Monocular Vision and Inertial Sensing , 2018, IEEE Robotics and Automation Letters.

[25]  Paulo Tabuada,et al.  An introduction to event-triggered and self-triggered control , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[26]  Shimon Whiteson,et al.  Counterfactual Multi-Agent Policy Gradients , 2017, AAAI.

[27]  Antonio Franchi,et al.  Decentralized motion control for cooperative manipulation with a team of networked mobile manipulators , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).