Augmented-Reality-Based Visualization of Navigation Data of Mobile Robots on the Microsoft Hololens - Possibilities and Limitations

The demand for mobile robots has rapidly increased in recent years due to the flexibility and high variety of application fields comparing to static robots. To deal with complex tasks such as navigation, they work with high amounts of different sensor data making it difficult to operate with for non-experts. To enhance user understanding and human robot interaction, we propose an approach to visualize the navigation stack within a cutting edge 3D Augmented Reality device -the Microsoft Hololens. Therefore, relevant navigation stack data including laser scan, environment map and path planing data are visualized in 3D within the head mounted device. Based on that prototype, we evaluate the Hololens in terms of computational capabilities and limitations for dealing with huge amount of real-time data. Results show that the Hololens is capable of a proper visualization of huge amounts of sensor data. We demonstrate a proper visualization of navigation stack data in 3D within the Hololens. However, there are limitations when transferring and displaying different kinds of data simultaneously.

[1]  Frank Steinicke,et al.  Comparison of Multimodal Heading and Pointing Gestures for Co-Located Mixed Reality Human-Robot Interaction , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[2]  David Bonyuet,et al.  Cooperative robot teleoperation through virtual reality interfaces , 2002, Proceedings Sixth International Conference on Information Visualisation.

[3]  J. Geoffrey Chase,et al.  Collaborating with a Mobile Robot: An Augmented Reality Multimodal Interface , 2008 .

[4]  Hooman Hedayati,et al.  Improving Collocated Robot Teleoperation with Augmented Reality , 2018, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[5]  Roland Siegwart,et al.  Introduction to Autonomous Mobile Robots , 2004 .

[6]  Rüdiger Dillmann,et al.  Using augmented reality to interact with an autonomous mobile platform , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[7]  Ernesto Damiani,et al.  Augmented reality technologies, systems and applications , 2010, Multimedia Tools and Applications.

[8]  Ramesh Raskar,et al.  Augmented Reality Visualization for Laparoscopic Surgery , 1998, MICCAI.

[9]  Simon Moosburner,et al.  Mixed Reality in Visceral Surgery d Evaluation of Intraoperative Use-cases Development of a Suitable Workflow an , 2017 .

[10]  Adam Rankin,et al.  Hologram stability evaluation for Microsoft HoloLens , 2017, Medical Imaging.

[11]  Volker Paelke,et al.  Augmented reality in the smart factory: Supporting workers in an industry 4.0. environment , 2014, Proceedings of the 2014 IEEE Emerging Technology and Factory Automation (ETFA).

[12]  Mathias Ciarlo Thorstensen Visualization of Robotic Sensor Data with Augmented Reality , 2017 .

[13]  Jörg Krüger,et al.  Spatial programming for industrial robots based on gestures and Augmented Reality , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Brice Gayet,et al.  3D visualization reduces operating time when compared to high-definition 2D in laparoscopic liver resection: a case-matched study , 2016, Surgical Endoscopy.

[15]  Timo Engelke,et al.  An augmented reality training platform for assembly and maintenance skills , 2013, Robotics Auton. Syst..

[16]  Jun Cheng,et al.  Mobile Cyber Physical Systems: Current Challenges and Future Networking Applications , 2018, IEEE Access.

[17]  Sungho Jo,et al.  Mapping System with Virtual Reality for Mobile Robot Teleoperation , 2018, 2018 18th International Conference on Control, Automation and Systems (ICCAS).

[18]  T. Igarashi,et al.  TouchMe : An Augmented Reality Based Remote Robot Manipulation , 2011 .

[19]  I. ClintHeyer Human-Robot Interaction and Future Industrial Robotics Applications , 2010 .

[20]  Soh-Khim Ong,et al.  Novel AR-based interface for human-robot interaction and visualization , 2014 .

[21]  Jan Guhl,et al.  Enabling Human-Robot-Interaction via Virtual and Augmented Reality in Distributed Control Systems , 2018 .

[22]  Abdulmotaleb El-Saddik,et al.  Technical Evaluation of HoloLens for Multimedia: A First Look , 2018, IEEE MultiMedia.

[23]  Adrien Coppens,et al.  Merging real and virtual worlds: An analysis of the state of the art and practical evaluation of Microsoft Hololens , 2017, ArXiv.