Programming contact tasks using a reality-based virtual environment integrated with vision

We present an integrated system in which an operator uses a simulated environment to program part-mating and contact tasks. Generation of models within this virtual environment is facilitated using a fast, occlusion tolerant, 3D grey-scale vision system which can recognize and accurately locate objects within the work site. A major goal of this work is to make robotic programming easy and intuitive for untrained users working with standard desktop hardware. Simulation offers the ease-of-use benefits of "programming by demonstration", coupled with the ability to create a programmer-friendly virtual environment. Within a simulated environment, it is also straightforward to track and interpret an operator's actions. The simulator models objects as polyhedra and implements full 3D contact dynamics. When a manipulation task is completed, local planning techniques are used to turn the virtual environment's motion sequence history into a set of robot motion commands capable of realizing the prescribed task.

[1]  Won S. Kim,et al.  The phantom robot: predictive displays for teleoperation with time delay , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[2]  J. De Schutter,et al.  A CAD-based contact force simulator as a learning tool for compliant motions , 1992, Proceedings of the 1992 IEEE International Symposium on Intelligent Control.

[3]  David G. Lowe,et al.  Shape indexing using approximate nearest-neighbour search in high-dimensional spaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  M. Doyon,et al.  On the implementation and performance of impedance control on position controlled robots , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[5]  Urs Ramer,et al.  An iterative procedure for the polygonal approximation of plane curves , 1972, Comput. Graph. Image Process..

[6]  K. Taylor,et al.  Australia's Telerobot On The Web , 1995 .

[7]  Richard P. Paul,et al.  Teleprogramming: Toward Delay-Invariant Remote Manipulation , 1992, Presence: Teleoperators & Virtual Environments.

[8]  Roy Featherstone,et al.  Robot Dynamics Algorithms , 1987 .

[9]  Craig Peter Sayers,et al.  Operator control of telerobotic systems for real world intervention , 1995 .

[10]  Sean Quinlan.  Real-time modification of collision-free paths , 1994 .

[11]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[12]  William H. Press,et al.  Numerical Recipes in FORTRAN - The Art of Scientific Computing, 2nd Edition , 1987 .

[13]  Oussama Khatib,et al.  Motion and force control of robot manipulators , 1986, Proceedings. 1986 IEEE International Conference on Robotics and Automation.

[14]  Dinesh K. Pai,et al.  Numerical solution of differential systems with algebraic inequalities arising in robot programming , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[15]  John F. Canny,et al.  Delivering real reality to the World Wide Web via telerobotics , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[16]  Lawrence W. Stark,et al.  Cooperative control of visual displays for telemanipulation , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[17]  Kazuo Tanie,et al.  Discrete-event-based planning and control of telerobotic part-mating process with communication delay and geometric uncertainty , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[18]  Pascal Fua,et al.  Computational strategies for object recognition , 1992, CSUR.

[19]  Gregory B. Newby,et al.  Virtual reality: Scientific and technological challenges , 1996 .

[20]  Richard P. Paul,et al.  An Operator Interface for Teleprogramming Employing Synthetic Fixtures , 1994, Presence: Teleoperators & Virtual Environments.

[21]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[22]  Kenneth Y. Goldberg,et al.  Desktop teleoperation via the World Wide Web , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[23]  Nathaniel I. Durlach,et al.  Virtual Reality: Scientific and Technological Challenges , 1994 .

[24]  Gerd Hirzinger,et al.  Sensor-based space robotics-ROTEX and its telerobotic features , 1993, IEEE Trans. Robotics Autom..

[25]  David Baraff,et al.  Fast contact force computation for nonpenetrating rigid bodies , 1994, SIGGRAPH.

[26]  W. Eric L. Grimson,et al.  Localizing Overlapping Parts by Searching the Interpretation Tree , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  David Baraff,et al.  Interactive simulation of solid rigid bodies , 1995, IEEE Computer Graphics and Applications.

[28]  R. Paul,et al.  Teleprogramming: overcoming communication delays in remote manipulation , 1990, 1990 IEEE International Conference on Systems, Man, and Cybernetics Conference Proceedings.

[29]  Howard Rheingold,et al.  Virtual Reality , 1991 .

[30]  Dinesh Manocha,et al.  I-COLLIDE: an interactive and exact collision detection system for large-scale environments , 1995, I3D '95.

[31]  Gerd Hirzinger,et al.  Learing Compliant Motions by Task-Demonstration in Virtual Environments , 1995, ISER.

[32]  Rakesh Mohan,et al.  Multidimensional Indexing for Recognizing Visual Shapes , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  David G. Lowe,et al.  Three-Dimensional Object Recognition from Single Two-Dimensional Images , 1987, Artif. Intell..

[34]  David G. Lowe,et al.  Fitting Parameterized Three-Dimensional Models to Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  David G. Lowe,et al.  Learning indexing functions for 3-D model-based object recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Susumu Tachi,et al.  Experimental Study on Remote Manipulation Using Virtual Reality , 1993, Presence: Teleoperators & Virtual Environments.