Multimedia Human-Computer Interaction for Presence and Exploration in a Telemuseum

It is not always feasible or practical to be present at a location of interest. Telepresence makes such possible. This paper presents the design and implementation of a telepresence system in which a mobile manipulator robot is utilized to explore a museum at a remote site, sending live video and audio on demand to a user at the local site. The user's sense of presence is also enhanced through the capability to manipulate objects remotely via the robot's gripper. The display system is a slightly modified, off-the-shelf mediascope, a television display system that can be worn like a pair of glasses. The head movements of the user are monitored using color tracking, and the displayed images are updated according to the motions of the head. This is achieved by interpreting the human head movements and transforming them into control signals for appropriate movements of the robot and its mounted color camera. Novel features of the presented system include utilization of human implicit commands, availability and cost-effectiveness of the system, feeling of presence, and intelligence fusion of the human user (where to go and what to do) with the robot's intelligence (local obstacle avoidance and navigation). The multimedia interface and control of the system are described in detail. Additionally, a number of research issues that require further investigation in order to realize attainable, high fidelity telepresence systems are discussed in this paper.

[1]  Martin Boyer.  Systems integration in telerobotics: case study: maintenance of electric power lines , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[2]  Lynette A. Jones,et al.  A Teleoperated Microsurgical Robot and Associated Virtual Environment for Eye Surgery , 1993, Presence: Teleoperators & Virtual Environments.

[3]  Blake Hannaford,et al.  Performance testing of passive communication and control in teleoperation with time delay , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[4]  Darwin G. Caldwell,et al.  Tele-presence: visual, audio and tactile feedback and control of a twin armed mobile robot , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[5]  Won S. Kim.  Graphical operator interface for space telerobotics , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[6]  Massimo Bergamasco,et al.  An arm exoskeleton system for teleoperation and virtual environments applications , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[7]  Kazuhiro Kosuge,et al.  Unified approach for teleoperation of virtual and real environment-manipulation based on reference dynamics , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[8]  John F. Canny,et al.  Delivering real reality to the World Wide Web via telerobotics , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[9]  Thomas B. Sheridan,et al.  Musings on Telepresence and Virtual Presence , 1992, Presence: Teleoperators & Virtual Environments.

[10]  I. Lin.  Interactive Environment Modeling and Vision-based Task Speciication for a Teleoperated Mobile Robot , 1995 .

[11]  Paolo Fiorini,et al.  Toward integrated operator interface for advanced teleoperation under time-delay , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).

[12]  Tzyh-Jong Tarn.  Intelligent planning and control for telerobotic operations , 1995, Proceedings of IECON '95 - 21st Annual Conference on IEEE Industrial Electronics.

[13]  Lawrence W. Stark,et al.  Sensing and Manipulation Problems in Endoscopic Surgery: Experiment, Analysis, and Observation , 1993, Presence: Teleoperators & Virtual Environments.

[14]  Gilles Clement,et al.  Computer aided teleoperation: from arm to vehicle control , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[15]  T B Sheridan,et al.  Teleoperator Performance with Varying Force and Visual Feedback , 1994, Human factors.

[16]  Tsutomu Miyasato,et al.  Virtual space teleconferencing "Communication with realistic sensations" , 1995, Proceedings 4th IEEE International Workshop on Robot and Human Communication.

[17]  Alberto Rovetta,et al.  Evaluation of human control in telerobotics by means of EMG , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).

[18]  Kenneth Y. Goldberg,et al.  Desktop teleoperation via the World Wide Web , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[19]  Grigore C. Burdea,et al.  Virtual reality and robotics in medicine , 1996, Proceedings 5th IEEE International Workshop on Robot and Human Communication. RO-MAN'96 TSUKUBA.

[20]  Richard P. Paul,et al.  Visual imagery for subsea teleprogramming , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[21]  Philip Amburn,et al.  A Prototype Visual and Audio Display , 1992, Presence: Teleoperators & Virtual Environments.

[22]  Sukhan Lee,et al.  Computer control of space-borne teleoperators with sensory feedback , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[23]  Francesco Mondada,et al.  "KhepOnTheWeb": An experimental demonstrator in telerobotics and virtual reality , 1997, Proceedings. International Conference on Virtual Systems and MultiMedia VSMM '97 (Cat. No.97TB100182).

[24]  Susumu Tachi,et al.  Experimental Study on Remote Manipulation Using Virtual Reality , 1993, Presence: Teleoperators & Virtual Environments.