Visual Communication with UAS: Recognizing Gestures from an Airborne Platform

Current tactical unmanned aerial systems receive their guidance and tasking information predominantly via radio links. To be able to communicate with these systems specific electronic devices are required. This work builds on the concept of visual communication of UAS to allow a person on ground commanding a nearby airborne vehicle to perform a specific reconnaissance task via gestures. A procedure to collect the necessary gestural command components is presented as well as a prototype image processing flow which is able to distinguish between neutral poses, static and dynamic 2D gestures. Prototype experiments prove the applicability of the proposed method on real life data from an airborne platform.

[1]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[2]  Morten Fjeld,et al.  How would you gesture navigate a drone?: a user-centered approach to control a drone , 2016, MindTrek.

[3]  Peter Stütz,et al.  Modelling Visual Communication with UAS , 2016, MESAS.

[4]  Cataldo Guaragnella,et al.  A Kinect-Based Gesture Recognition Approach for a Natural Human Robot Interface , 2015 .

[5]  Sergio Escalera,et al.  Challenges in multimodal gesture recognition , 2016, J. Mach. Learn. Res..

[6]  James A. Landay,et al.  Drone & me: an exploration into natural human-drone interaction , 2015, UbiComp.

[7]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[8]  Michael Felsberg,et al.  Accurate Scale Estimation for Robust Visual Tracking , 2014, BMVC.

[9]  Tae-Kyun Kim,et al.  Unconstrained Monocular 3D Human Pose Estimation by Action Detection and Cross-Modality Regression Forest , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Martin Hitz,et al.  Exploring User-Defined Gestures and Voice Commands to Control an Unmanned Aerial Vehicle , 2016, INTETAIN.

[11]  Nassir Navab,et al.  Human skeleton tracking from depth data using geodesic distances and optical flow , 2012, Image Vis. Comput..

[12]  Aurélien Garivier,et al.  On the Complexity of Best-Arm Identification in Multi-Armed Bandit Models , 2014, J. Mach. Learn. Res..

[13]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).