Multimodal user interface for remote object exploration with sparse sensory data

This paper describes the design and current state of development of a multimodal user interface enabling exploration of remote environments by means of a teleoperated robot. Human-robot interaction consists of visual and tactile feedback to the user and gesture-based teleoperation of the robot. The user interface includes surface reconstruction of the remote environment integrating data sampled by tactile and proximity sensors, cutaneous perception of distance information through the vibrotactile actuators integrated in a virtual reality glove (the CyberTouch, from Immersion Corp., Inc.), and an optional streaming of video information for bandwidth-rich settings. Preliminary results show that the VR glove enables the operator to drive the exploration of the remote environment in a natural fashion and to react promptly to the sensed information.

[1]  Gerd Hirzinger,et al.  Learning techniques in a dataglove based telemanipulation system for the DLR hand , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[2]  Mark R. Cutkosky,et al.  Development and Testing of a Telemanipulation System with Arm and Hand Motion , 2000 .

[3]  Hyeong In Choi,et al.  The Medial Axis Transform , 2002, Handbook of Computer Aided Geometric Design.

[4]  Sunghee Choi,et al.  The power crust, unions of balls, and the medial axis transform , 2001, Comput. Geom..

[5]  R. Klatzky,et al.  There's more to touch than meets the eye: The salience of object attributes for haptics with and without vision. , 1987 .

[6]  Peter K. Allen,et al.  Integrating Vision and Touch for Object Recognition Tasks , 1988, Int. J. Robotics Res..

[7]  Brian Mirtich,et al.  V-Clip: fast and robust polyhedral collision detection , 1998, TOGS.

[8]  Gabriel Taubin,et al.  The ball-pivoting algorithm for surface reconstruction , 1999, IEEE Transactions on Visualization and Computer Graphics.

[9]  Sing Bing Kang,et al.  Robot instruction by human demonstration , 1995 .

[10]  Masaru Uchiyama,et al.  Verification of an advanced space teleoperation system using Internet , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[11]  Klaus Landzettel,et al.  Teleoperating space robots. Impact for the design of industrial robots , 1997, ISIE '97 Proceeding of the IEEE International Symposium on Industrial Electronics.

[12]  Mark R. Cutkosky,et al.  Calibration and Mapping of a Human Hand for Dexterous Telemanipulation , 2000, Dynamic Systems and Control: Volume 2.

[13]  Craig Sayers,et al.  Remote control robotics , 1998 .

[14]  Michele Amoretti,et al.  A software framework based on real-time CORBA for telerobotic systems , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Tony DeRose,et al.  Surface reconstruction from unorganized points , 1992, SIGGRAPH.

[16]  Claudio Melchiorri,et al.  Design of a Three-Finger Gripper for Intra-Vehicular Robotic Manipulation , 1998 .

[17]  Francesco Zanichelli,et al.  Investigation of polyhedral shape representations and connectionist techniques in haptic object recognition , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).