Effects of Haptic and 3D Audio Feedback on Operator Performance and Workload for Quadrotor UAVs in Indoor Environments

Unmanned aerial vehicles (UAVs) have many potential applications in indoor environments. However, limited visual feedback makes it difficult to pilot UAVs in cluttered and enclosed spaces. Haptic feedback combined with visual feedback has been shown to reduce the number of collisions of UAVs in indoor environments, but has generally resulted in an increase in the mental workload of the operator. This paper investigates the potential of combining novel haptic and 3D audio feedback to provide additional information to operators of UAVs to improve performance and reduce workload. Two haptic feedback and two 3D audio feedback algorithms are presented and tested in a simulation-based human subject experiment. Operator workload is quantified using standard measures and a novel application of behavioral entropy. Experimental results indicate that 3D haptic feedback improved UAV pilot performance. Pilot workload was also improved for one of the haptic algorithms in one of the control directions (lateral). The 3D audio feedback algorithms investigated in this study neither improved nor degraded pilot performance.

[1]  Ellen Haas,et al.  Multimodal displays to enhance human robot interaction on-the-move , 2007 .

[2]  Steven Lake Waslander,et al.  Aerodynamics and control of autonomous quadrotor helicopters in aggressive maneuvering , 2009, 2009 IEEE International Conference on Robotics and Automation.

[3]  Andreas Birk,et al.  An industrial application of behavior-oriented robotics , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[4]  Dale A. Carnegie,et al.  The design of a mobile autonomous robot for indoor security applications , 2004 .

[5]  Antonio Franchi,et al.  An evaluation of haptic cues on the tele-operator's perceptual awareness of multiple UAVs' environments , 2011, 2011 IEEE World Haptics Conference.

[6]  Gaurav S. Sukhatme,et al.  Haptic control of a mobile robot: a user study , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Yoram Koren,et al.  Real-time obstacle avoidance for fast mobile robots in cluttered environments , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[8]  René van Paassen,et al.  Collision avoidance for a remotely-operated helicopter using haptic feedback , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[9]  Arcangelo Distante,et al.  An Autonomous Mobile Robotic System for Surveillance of Indoor Environments , 2010 .

[10]  M. M. van Paassen,et al.  Design and Evaluation of a Haptic Flight Director , 2006 .

[11]  Byung-Cheol Min,et al.  Development of a Micro Quad-Rotor UAV for Monitoring an Indoor Environment , 2009, FIRA RoboWorld Congress.

[12]  Mark B. Colton,et al.  Haptic collision avoidance for a remotely operated quadrotor UAV in indoor environments , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[13]  Randal W. Beard,et al.  Multi-vehicle dynamics and control for aerial recovery of micro air vehicles , 2011 .

[14]  Wolfram Burgard,et al.  Towards a navigation system for autonomous indoor flying , 2009, 2009 IEEE International Conference on Robotics and Automation.

[15]  P. Hancock,et al.  Human Mental Workload , 1988 .

[16]  Christer Ericson,et al.  Real-Time Collision Detection , 2004 .

[17]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[18]  Jee-Hwan Ryu,et al.  Haptic interface for intuitive teleoperation of wheeled and tracked vehicles , 2011, 2011 IEEE World Haptics Conference.

[19]  Christopher A. Mattson,et al.  A Differentially Driven Flapping Wing Mechanism for Force Analysis and Trajectory Optimization , 2012 .

[20]  Robert S. Bolia,et al.  Aurally Aided Visual Search in Three-Dimensional Space , 1999, Hum. Factors.

[21]  René van Paassen,et al.  Haptic Feedback for UAV Tele-operation - Force offset and spring load modification , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[22]  David D. Woods,et al.  Envisioning human-robot coordination in future operations , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[23]  Michael A. Goodrich,et al.  Incorporating a robot into an autism therapy team , 2012, IEEE Intelligent Systems.

[24]  William Cyrus Navidi,et al.  Statistics for Engineers and Scientists , 2004 .

[25]  Hudson D Graham,et al.  Effect of Auditory Peripheral Displays On Unmanned Aerial Vehicle Operator Performance , 2008 .

[26]  Qi Wang,et al.  System Design and Control Research for New Quadrotor Searching and Saving Aircraft , 2012 .

[27]  Erwin Roeland Boer.  Behavioral Entropy as an Index of Workload , 2000 .

[28]  Michael A. Goodrich,et al.  Behavioral Entropy in Human-Robot Interaction , 2004 .

[29]  Timothy W. McLain,et al.  Utilizing an improved rotorcraft dynamic model in state estimation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  Takuya Nojima,et al.  Cockpit display using tactile sensation , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[31]  Aníbal Ollero,et al.  Multimodal Interface Technologies for UAV Ground Control Stations , 2010, J. Intell. Robotic Syst..

[32]  Stephen A. Brewster,et al.  Evaluation of multimodal graphs for blind people , 2003, Universal Access in the Information Society.

[33]  M.M. Balas,et al.  Driver Assisting by Inverse Time to Collision , 2006, 2006 World Automation Congress.

[34]  Frans C. T. van der Helm,et al.  Effect of haptic feedback in a trajectory following task with an unmanned aerial vehicle , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[35]  Erwin R. Boer,et al.  Development of a steering entropy method for evaluating driver workload , 1999 .

[36]  Tianmiao Wang,et al.  The design of multimodal human-machine interface for teleoperation , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[37]  Joshua D. Hoffman,et al.  Collision warning design to mitigate driver distraction , 2004, CHI.

[38]  W. T. Nelson,et al.  Target Acquisition With UAVs: Vigilance Displays and Advanced Cuing Interfaces , 2005, Hum. Factors.

[39]  Marko Munih,et al.  Psychophysiological responses to different levels of cognitive and physical workload in haptic interaction , 2010, Robotica.

[40]  Hendrik A. H. C. van Veen,et al.  A Tactile Cockpit Instrument Supports the Control of Self-Motion During Spatial Disorientation , 2006, Hum. Factors.

[41]  Timothy W. McLain,et al.  Relative navigation and control of a hexacopter , 2012, 2012 IEEE International Conference on Robotics and Automation.

[42]  Peter I. Corke,et al.  A novel approach to haptic tele-operation of aerial robot vehicles , 2010, 2010 IEEE International Conference on Robotics and Automation.

[43]  René van Paassen,et al.  Haptic interface in UAV tele-operation using force-stiffness feedback , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.