Utilizing Glove-Based Gestures and a Tactile Vest Display for Covert Communications and Robot Control

Abstract : Dismounted Soldiers consistently experience heavy cognitive and visual workload, particularly during navigation and patrol, and under conditions of high stress and time pressure. This report describes an investigation of the potential of gestural controls and a tactile vest display expected to reduce cognitive, physical, and temporal demands as they relate to covert communications and robot control. Data were collected on 31 Soldiers. Twelve Soldiers were from the Explosive Ordnance Disposal military occupational specialty and had extensive experience with robot control. In this experiment-based evaluation, the task demands in a typical rural reconnaissance patrol were deconstructed into specific tasks involving a platoon leader role, a squad leader role, and a robot controller role. Results showed that use of an instrumented glove and a tactile vest display for covert communications was associated with a higher average percentage and faster average speed of signal detections when compared to traditional hand and arm signals. Also, glove-based control of robot maneuvers was compared with a handheld controller. While the glove controller was not associated with better performance outcomes, Soldiers expressed high regard for the concept, and provided many suggestions for further development of gestural controls for both covert communications and robot control.

[1]  Charles J. Cohen,et al.  A control theoretic method for categorizing visual imagery as human motion behaviors , 2005, 34th Applied Imagery and Pattern Recognition Workshop (AIPR'05).

[2]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[3]  M. Urban,et al.  Recognition of arm gestures using multiple orientation sensors: repeatability assessment , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[4]  Regina A. Pomranky,et al.  Determination of Mental Workload During Operation of Multiple Unmanned Systems , 2007 .

[5]  Yale Song,et al.  Multi-signal gesture recognition using temporal smoothing hidden conditional random fields , 2011, Face and Gesture 2011.

[6]  Diane Kuhl. Mitchell,et al.  Workload Analysis of the Crew of the Abrams V2 SEP: Phase I Baseline IMPRINT Model , 2009 .

[7]  Holger Kenn,et al.  A glove-based gesture interface for wearable computing applications , 2007 .

[8]  Marjorie Skubic,et al.  Using spatial language in a human-robot dialog , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[9]  James M. Keller,et al.  Spatial relations for tactical robot navigation , 2001, SPIE Defense + Commercial Sensing.

[10]  Bruce J. P. Mortimer,et al.  Implementing Effective Tactile Symbology for Orientation and Navigation , 2011, HCI.

[11]  James M. Keller,et al.  Generating linguistic spatial descriptions from sonar readings using the histogram of forces , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[12]  Kenneth Wauchope,et al.  Eucalyptus: Integrating Natural Language Input with a Graphical User Interface , 1994 .

[13]  Yale Song,et al.  Continuous body and hand gesture recognition for natural human-computer interaction , 2012, TIIS.

[14]  Christopher D. Wickens,et al.  Multiple resources and performance prediction , 2002 .

[15]  Linda R. Elliott,et al.  Development of Tactile and Haptic Systems for U.S. Infantry Navigation and Communication , 2011, HCI.

[16]  Hyeran Byun,et al.  Visual recognition of aircraft marshalling signals using gesture phase analysis , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[17]  Ming C. Leu,et al.  Linguistic properties based on American Sign Language isolated word recognition with artificial neural networks using a sensory glove and motion tracker , 2007, Neurocomputing.

[18]  Christopher D. Wickens,et al.  Multiple Resources and Mental Workload , 2008, Hum. Factors.

[19]  J. Gregory Trafton,et al.  Spatial Representation and Reasoning for Human-Robot Collaboration , 2007, AAAI.

[20]  Alan C. Schultz,et al.  Using a natural language and gesture interface for unmanned vehicles , 2000, Defense, Security, and Sensing.

[21]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[22]  Marion G. Ceruti,et al.  Gesture-Directed Sensor-Information Fusion for Communication in Hazardous Environments , 2010 .

[23]  Christian B. Carstens,et al.  Comparison of Army Hand and Arm Signals to a Covert Tactile Communication System in a Dynamic Environment , 2006 .

[24]  Linda R. Elliott,et al.  Tactile Cuing to Augment Multisensory Human-Machine Interaction , 2015 .

[25]  Sarangi P. Parikh,et al.  Can robots recognize common Marine gestures? , 2010, 2010 42nd Southeastern Symposium on System Theory (SSST).

[26]  Jwu-Sheng Hu,et al.  Self-balancing control and manipulation of a glove puppet robot on a two-wheel mobile platform , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  J. Gregory Trafton,et al.  Finding the FOO: a pilot study for a multimodal interface , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[28]  Thomas S. Huang,et al.  Gesture modeling and recognition using finite state machines , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[29]  Eric Fujiwara,et al.  Development of a glove-based optical fiber sensor for applications in human-robot interaction , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[30]  Yale Song,et al.  Tracking body and hands for gesture recognition: NATOPS aircraft handling signals database , 2011, Face and Gesture 2011.

[31]  J.B.F. van Erp.  Tactile displays for navigation and orientation : perception and behaviour , 2007 .

[32]  Bandman El.  Protection of human subjects. , 1980, JAMA.

[33]  张哉根,et al.  Leu-M , 1991 .

[34]  Linda R. Elliott,et al.  Overview of Meta-analyses Investigating Vibrotactile versus Visual Display Options , 2009, HCI.

[35]  Mu-Chun Su,et al.  A hand-gesture-based control interface for a car-robot , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[36]  Björn Karlsson,et al.  A glove equipped with finger flexion sensors as a command generator used in a fuzzy control system , 1998, IEEE Trans. Instrum. Meas..

[37]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .