Adaptive View Management for Drone Teleoperation in Complex 3D Structures

Drone navigation in complex environments poses many problems to teleoperators. Especially in 3D structures like buildings or tunnels, viewpoints are often limited to the drone's current camera view, nearby objects can be collision hazards, and frequent occlusion can hinder accurate manipulation. To address these issues, we have developed a novel interface for teleoperation that provides a user with environment-adaptive viewpoints that are automatically configured to improve safety and smooth user operation. This real-time adaptive viewpoint system takes robot position, orientation, and 3D pointcloud information into account to modify user-viewpoint to maximize visibility. Our prototype uses simultaneous localization and mapping (SLAM) based reconstruction with an omnidirectional camera and we use resulting models as well as simulations in a series of preliminary experiments testing navigation of various structures. Results suggest that automatic viewpoint generation can outperform first and third-person view interfaces for virtual teleoperators in terms of ease of control and accuracy of robot operation.

[1]  David W. Hainsworth,et al.  Teleoperation User Interfaces for Mining Robotics , 2001, Auton. Robots.

[2]  Ken Shoemake,et al.  Animating rotation with quaternion curves , 1985, SIGGRAPH.

[3]  Yasushi Yagi,et al.  Omnidirectional imaging with hyperboloidal projection , 1993, Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '93).

[4]  Roland Siegwart,et al.  Vision-Controlled Micro Flying Robots: From System Design to Autonomous Navigation and Mapping in GPS-Denied Environments , 2014, IEEE Robotics & Automation Magazine.

[5]  Daniel Cremers,et al.  Camera-based navigation of a low-cost quadrocopter , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[7]  Charles R. Dyer,et al.  Interpolating view and scene motion by dynamic view morphing , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[8]  Greg Mori,et al.  HRI in the sky: Creating and commanding teams of UAVs with a vision-mediated gestural interface , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Naokazu Yokoya,et al.  [Paper] Free-viewpoint Mobile Robot Teleoperation Interface Using View-dependent Geometry and Texture , 2014 .

[10]  Atsushi Yamashita,et al.  A decoupled virtual camera using spherical optical flow , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[11]  Takeo Igarashi,et al.  A teleoperating interface for ground vehicles using autonomous flying cameras , 2013, 2013 23rd International Conference on Artificial Reality and Telexistence (ICAT).

[12]  Lennart E. Nacke,et al.  More than a feeling: Measurement of sonic user experience and psychophysiology in a first-person shooter game , 2010, Interact. Comput..

[13]  Kiyoshi Kiyokawa,et al.  Dynamic text management for see-through wearable and heads-up display systems , 2013, IUI '13.

[14]  Daniel Cremers,et al.  LSD-SLAM: Large-Scale Direct Monocular SLAM , 2014, ECCV.

[15]  F. Matsuno,et al.  Study on effective camera images for mobile robot teleoperation , 2004, RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759).

[16]  Shoichi Maeyama,et al.  View Point Decision Algorithm for an Autonomous Robot to Provide Support Images in the Operability of a Teleoperated Robot , 2016 .

[17]  Dieter Schmalstieg,et al.  Image-driven view management for augmented reality browsers , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).