Single-Handed Gesture UAV Control for First Responders - A Usability and Performance User Study

Unmanned aerial vehicles (UAVs) have increased in popularity in recent years and are now involved in many activities, professional and otherwise. First responders, those teams and individuals who are the first to respond in crisis situations, have been using UAVs to assist them in locating victims and identifying hazards without endangering human personnel needlessly. However, professional UAV controllers tend to be heavy and cumbersome, requiring both hands to operate. First responders, on the other hand, often need to carry other important equipment and need to keep their hands free during a mission. This work considers enabling first responders to control UAVs corresponding author †http://vcl.iti.gr/ WiP Paper – Technologies for First Responders Proceedings of the 17th ISCRAM Conference – Blacksburg, VA, USA May 2020 Amanda Lee Hughes, Fiona McNeill and Christopher Zobel, eds. 937 Konstantoudakis et al. Gesture UAV Control for First Responders with single-handed gestures, freeing their other hand and reducing their encumbrance. Two sets of gesture UAV controls are presented and implemented in a simulated environment, and a two-part user study is conducted: the first part assesses the comfort of each gesture and their intuitive association with basic flight control concepts; and the second evaluates two di erent modes of gesture control in a population of users including both genders, and first responders as well as members of the general populace. The results, consisting of both objective and subjective measurements, are discussed, hindrances and problems are identified, and directions of future work and research are mapped out.

[1]  Junsong Yuan,et al.  3D Hand Shape and Pose Estimation From a Single RGB Image , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Giuseppe Loianno,et al.  Human Gaze-Driven Spatial Tasking of an Autonomous MAV , 2019, IEEE Robotics and Automation Letters.

[3]  Petros Daras,et al.  A Deep Learning Approach for Analyzing Video and Skeletal Features in Sign Language Recognition , 2018, 2018 IEEE International Conference on Imaging Systems and Techniques (IST).

[4]  Yuzhong Shen,et al.  Traditional Vs Gesture Based UAV Control , 2018, Advances in Intelligent Systems and Computing.

[5]  Kin Hong Wong,et al.  Robust Hand Gesture Input Using Computer Vision, Inertial Measurement Unit (IMU) and Flex Sensors , 2018, 2018 IEEE International Conference on Mechatronics, Robotics and Automation (ICMRA).

[6]  Otmar Hilliges,et al.  Cross-Modal Deep Variational Hand Pose Estimation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  Truong-Huy D. Nguyen,et al.  Hand Gesture Controlled Drones: An Open Source Library , 2018, 2018 1st International Conference on Data Intelligence and Security (ICDIS).

[8]  Jennifer Lee,et al.  Communicating Robot Motion Intent with Augmented Reality , 2018, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[9]  Sergio Escalera,et al.  Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Matthias Nießner,et al.  Matterport3D: Learning from RGB-D Data in Indoor Environments , 2017, 2017 International Conference on 3D Vision (3DV).

[11]  Erica L. Meszaros,et al.  Analysis of a gesture-based interface for UAV flight path generation , 2017, 2017 International Conference on Unmanned Aircraft Systems (ICUAS).

[12]  Ashish Kapoor,et al.  AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles , 2017, FSR.

[13]  Thomas Brox,et al.  Learning to Estimate 3D Hand Pose from Single RGB Images , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  Vijay Kumar,et al.  Autonomous Navigation and Mapping for Inspection of Penstocks and Tunnels With MAVs , 2017, IEEE Robotics and Automation Letters.

[15]  Pavlo Molchanov,et al.  Online Detection and Classification of Dynamic Hand Gestures with Recurrent 3D Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Daniel Thalmann,et al.  Robust 3D Hand Pose Estimation in Single Depth Images: From Single-View CNN to Multi-View CNNs , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Vijay Kumar,et al.  Cooperative localization and mapping of MAVs using RGB-D sensors , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Luca Maria Gambardella,et al.  Human-swarm interaction using spatial gestures , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Frank Weichert,et al.  Analysis of the Accuracy and Robustness of the Leap Motion Controller , 2013, Sensors.

[20]  Joseph J. LaViola,et al.  Exploring 3d gesture metaphors for interaction with unmanned aerial vehicles , 2013, IUI '13.

[21]  Darius Burschka,et al.  Toward a Fully Autonomous UAV: Research Platform for Indoor and Outdoor Urban Search and Rescue , 2012, IEEE Robotics & Automation Magazine.

[22]  Kazuya Yoshida,et al.  Collaborative mapping of an earthquake‐damaged building via ground and aerial robots , 2012, J. Field Robotics.

[23]  Lorenzo Marconi,et al.  Impedance control of an aerial manipulator , 2012, 2012 American Control Conference (ACC).

[24]  Ehud Sharlin,et al.  Collocated interaction with flying robots , 2011, 2011 RO-MAN.

[25]  Sebastian Thrun,et al.  A Gesture Based Interface for Human-Robot Interaction , 2000, Auton. Robots.

[26]  Michael Vande Weghe,et al.  An architecture for gesture-based control of mobile robots , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[27]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  J. Norberto Pires,et al.  High-Level Programming for Industrial Robotics : using Gestures , Speech and Force Control , 2008 .