Baxter's Homunculus: Virtual Reality Spaces for Teleoperation in Manufacturing

We demonstrate a low-cost telerobotic system that leverages commercial virtual reality (VR) technology and integrates it with existing robotics control infrastructure. The system runs on a commercial gaming engine using off-the-shelf VR hardware and can be deployed on multiple network architectures. The system is based on the homunculus model of mind wherein we embed the user in a VR control room. The control room allows for multiple sensor displays, and dynamic mapping between the user and robot. This dynamic mapping allows for selective engagement between the user and the robot. We compared our system with state-of-the-art automation algorithms and standard VR-based telepresence systems by performing a user study. The study showed that new users were faster and more accurate than the automation or a direct telepresence system. We also demonstrate that our system can be used for pick and place, assembly, and manufacturing tasks.

[1]  R. L. Gregory.  The Oxford companion to the mind , 1989 .

[2]  D. Dennett,et al.  The Nature of Consciousness , 2006 .

[3]  Kay M. Stanney,et al.  Postural instability induced by virtual reality exposure: Development of a certification protocol , 1996, Int. J. Hum. Comput. Interact..

[4]  Frederick P. Brooks,et al.  Moving objects in space: exploiting proprioception in virtual-environment interaction , 1997, SIGGRAPH.

[5]  Jonathan Kofman,et al.  Teleoperation of a robot manipulator using a vision-based human-robot interface , 2005, IEEE Transactions on Industrial Electronics.

[6]  Satoshi Kagami,et al.  Humanoid teleoperation for whole body manipulation , 2008, 2008 IEEE International Conference on Robotics and Automation.

[7]  Florian Schmidt,et al.  Multimodal telepresent control of DLR's Rollin' JUSTIN , 2009, 2009 IEEE International Conference on Robotics and Automation.

[8]  Martin Buss,et al.  A Survey of Environment-, Operator-, and Task-adapted Controllers for Teleoperation Systems , 2010 .

[9]  Edwin Olson,et al.  LCM: Lightweight Communications and Marshalling , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Martin Kraus,et al.  Using a Graphics Turing Test to Evaluate the Effect of Frame Rate and Motion Blur on Telepresence of Animated Objects , 2013, GRAPP/IVAPP.

[11]  Gregory D. Hager,et al.  Adjutant: A framework for flexible human-machine collaborative systems , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Z. Li,et al.  Teleoperation control of Baxter robot using body motion tracking , 2014, 2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems (MFI).

[13]  Long Cheng,et al.  Shared control for teleoperation enhanced by autonomous obstacle avoidance of robot manipulator , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[14]  Sven Behnke,et al.  Intuitive bimanual telemanipulation under communication restrictions by immersive 3D visualization and motion tracking , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[15]  Jan Peters,et al.  First-person tele-operation of a humanoid robot , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[16]  Daniela Rus,et al.  Using Vision for Pre- and Post-grasping Object Localization for Soft Hands , 2016, ISER.