Gesture based Human-Swarm Interactions for Formation Control using interpreters

We propose a novel Human-Swarm Interaction (HSI) framework which enables the user to control a swarm shape and formation. The user commands the swarm utilizing just arm gestures and motions which are recorded by an off-the-shelf wearable armband. We propose a novel interpreter system, which acts as an intermediary between the user and the swarm to simplify the user's role in the interaction. The interpreter takes in a high level input drawn using gestures by the user, and translates it into low level swarm control commands. This interpreter employs machine learning, Kalman filtering and optimal control techniques to translate the user input into swarm control parameters. A notion of Human Interpretable dynamics is introduced, which is used by the interpreter for planning as well as to provide feedback to the user. The dynamics of the swarm are controlled using a novel decentralized formation controller based on distributed linear iterations and dynamic average consensus. The framework is demonstrated theoretically as well as experimentally in a 2D environment, with a human controlling a swarm of simulated robots in real time.

[1]  Jianghai Hu,et al.  On the Value Functions of the Discrete-Time Switched LQR Problem , 2009, IEEE Transactions on Automatic Control.

[2]  Luca Maria Gambardella,et al.  Wearable multi-modal interface for human multi-robot interaction , 2016, 2016 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).

[3]  Magnus Egerstedt,et al.  Haptic interactions with multi-robot swarms using manipulability , 2015, HRI 2015.

[4]  Sonia Martínez,et al.  Discrete-time dynamic average consensus , 2010, Autom..

[5]  Mac Schwager,et al.  Brain-Swarm Interface (BSI): Controlling a Swarm of Robots with Brain and Eye Signals from an EEG Headset , 2016, ArXiv.

[6]  Katia P. Sycara,et al.  Human Interaction With Robot Swarms: A Survey , 2016, IEEE Transactions on Human-Machine Systems.

[7]  Radhika Nagpal,et al.  Programmable self-assembly in a thousand-robot swarm , 2014, Science.

[8]  Emilio Frazzoli,et al.  A Dynamical Queue Approach to Intelligent Task Management for Human Operators , 2012, Proceedings of the IEEE.

[9]  Paul A. Beardsley,et al.  Gesture based human - Multi-robot swarm interaction and its application to an interactive display , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Paul A. Beardsley,et al.  Image and animation display with multiple mobile robots , 2012, Int. J. Robotics Res..

[11]  Wolfram Burgard,et al.  Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) , 2005 .

[12]  Gordon F. Royle,et al.  Algebraic Graph Theory , 2001, Graduate texts in mathematics.

[13]  Aamodh Suresh.  Body-Swarm Interface (BoSI) : Controlling robotic swarms using human bio-signals , 2016 .

[14]  Jorge Cortes,et al.  Distributed Control of Robotic Networks: A Mathematical Approach to Motion Coordination Algorithms , 2009 .

[15]  Antonio Franchi,et al.  Shared Control : Balancing Autonomy and Human Assistance with a Group of Quadrotor UAVs , 2012, IEEE Robotics & Automation Magazine.

[16]  Reza Olfati-Saber,et al.  Consensus and Cooperation in Networked Multi-Agent Systems , 2007, Proceedings of the IEEE.

[17]  Luca Maria Gambardella,et al.  Human-swarm interaction using spatial gestures , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  P. Olver.  Nonlinear Systems , 2013 .

[19]  Mac Schwager,et al.  Kinematic multi-robot manipulation with no communication using force feedback , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Stephen P. Boyd,et al.  Fast linear iterations for distributed averaging , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[21]  Antonio Franchi.  Human-Collaborative Schemes in the Motion Control of Single and Multiple Mobile Robots , 2017 .

[22]  Jie Lin,et al.  Coordination of groups of mobile autonomous agents using nearest neighbor rules , 2003, IEEE Trans. Autom. Control..

[23]  Jorge Cortes,et al.  Global formation-shape stabilization of relative sensing networks , 2009, 2009 American Control Conference.

[24]  Muaz A. Niazi,et al.  Technical Problems With "Programmable self-assembly in a thousand-robot swarm" , 2014, ArXiv.

[25]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[26]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.