Invited review: the synergy between virtual reality and robotics

To a large extent the robotics and the newer virtual reality (VR) research communities have been working in isolation. This article reviews three areas where integration of the two technologies can be beneficial. First, we consider VR-enhanced CAD design, robot programming, and plant layout simulation. Subsequently, we discuss how VR is being used in supervisory teleoperation, for single operator-single robot systems, single operator multiplexed to several slave robots, and collaborative control of a single robot by multiple operators. Here VR can help overcome problems related to poor visual feedback as well as system instabilities due to time delays. Finally, we show how robotics can be beneficial to VR in general, since robots can serve as force feedback interfaces to the simulation. Newer back-drivable manipulators offer increased safety for the user that closely interacts with the robot. The synergy between the fields of robotics and virtual reality is expected to grow in years to come.

[1]  Wyatt S. Newman,et al.  Virtual testing of agile manufacturing software using 3D graphical simulation , 1997, Proceedings of International Conference on Robotics and Automation.

[2]  Lawrence W. Stark,et al.  Model-Based Supervisory Control in Telerobotics , 1996, Presence: Teleoperators & Virtual Environments.

[3]  William R. Mark,et al.  UNC-CH Force Feedback Library , 1994 .

[4]  Gerd Hirzinger,et al.  ROTEX-the first remotely controlled robot in space , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[5]  Rajit Gadh,et al.  Multimodal interface for a virtual reality based computer aided design system , 1997, Proceedings of International Conference on Robotics and Automation.

[6]  Michael J. McDonald,et al.  Virtual collaborative control to improve intelligent robotic system efficiency and quality , 1997, Proceedings of International Conference on Robotics and Automation.

[7]  Geb Thomas,et al.  Virtual Tools for Supervisory and Collaborative Control of Robots , 1997, Presence: Teleoperators & Virtual Environments.

[8]  Thomas H. Massie,et al.  The PHANToM Haptic Interface: A Device for Probing Virtual Objects , 1994 .

[9]  Kam S. Tso,et al.  Task lines and motion guides , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[10]  William A. McNeely,et al.  Robotic graphics: a new approach to force feedback for virtual reality , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[11]  Qingping Lin,et al.  Virtual tele-operation of underwater robots , 1997, Proceedings of International Conference on Robotics and Automation.

[12]  Louis B. Rosenberg.  “Virtual fixtures”: perceptual overlays enhance operator performance in telepresence tasks , 1994 .

[13]  Henry Sowizral,et al.  Implementation of Dynamic Robotic Graphics For a Virtual Control Panel , 1997, Presence: Teleoperators & Virtual Environments.

[14]  J. Edward Colgate,et al.  Passive robots and haptic displays based on nonholonomic elements , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[15]  Abderrahmane Kheddar,et al.  Parallel multi-robots long distance teleoperation , 1997, 1997 8th International Conference on Advanced Robotics. Proceedings. ICAR'97.

[16]  Rakesh Gupta,et al.  Experiments Using Multimodal Virtual Environments in Design for Assembly Analysis , 1997, Presence: Teleoperators & Virtual Environments.

[17]  Tetsuo Kotoku.  A Predictive Display With Force Feedback And Its Application To Remote Manipulation System With Transmission Time Delay , 1992, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  John R. Wilson,et al.  Manufacturing Operations in Virtual Environments (MOVE) , 1995, Presence: Teleoperators & Virtual Environments.

[19]  Elizabeth M. Wenzel,et al.  Localization with non-individualized virtual acoustic display cues , 1991, CHI.

[20]  Gregory B. Newby,et al.  Virtual reality: Scientific and technological challenges , 1996 .

[21]  James A. Rehg.  Introduction to Robotics in CIM Systems , 1992 .

[22]  Won S. Kim,et al.  Virtual Reality Calibration and Preview/Predictive Displays for Telerobotics , 1996, Presence: Teleoperators & Virtual Environments.

[23]  Won S. Kim,et al.  The phantom robot: predictive displays for teleoperation with time delay , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[24]  Takao Kakizaki,et al.  Task world reality for human and robot system. A multimodal teaching advisor and its implementation , 1996, Proceedings 5th IEEE International Workshop on Robot and Human Communication. RO-MAN'96 TSUKUBA.

[25]  Abderrahmane Kheddar,et al.  Detection of discrepancies and sensory-based recovery for virtual reality based telemanipulation systems , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[26]  John F. Canny,et al.  Delivering real reality to the World Wide Web via telerobotics , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[27]  Carolina Cruz-Neira,et al.  Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE , 2023 .

[28]  Christian Blume,et al.  Programming languages for industrial robots , 1986, Symbolic computation.

[29]  Michitaka Hirose,et al.  Providing force feedback in virtual environments , 1995, IEEE Computer Graphics and Applications.

[30]  Richard P. Paul,et al.  An Operator Interface for Teleprogramming Employing Synthetic Fixtures , 1994, Presence: Teleoperators & Virtual Environments.

[31]  Grigore C. Burdea,et al.  Force and Touch Feedback for Virtual Reality , 1996 .

[32]  Susumu Tachi,et al.  Experimental Study on Remote Manipulation Using Virtual Reality , 1993, Presence: Teleoperators & Virtual Environments.

[33]  Paul Richard,et al.  Effect of Frame Rate and Force Feedback on Virtual Object Manipulation , 1996, Presence: Teleoperators & Virtual Environments.

[34]  Elizabeth M. Wenzel,et al.  Localization in Virtual Acoustic Displays , 1992, Presence: Teleoperators & Virtual Environments.

[35]  Susumu Tachi,et al.  VR Reborn , 1998, IEEE Computer Graphics and Applications.