Human-Machine Collaborative Systems for Microsurgical Applications

We describe our current progress in developing Human-Machine Collaborative Systems (HMCSs) for microsurgical applications such as vitreo-retinal eye surgery. Three specific problems considered here are (1) developing of systems tools for describing and implementing an HMCS, (2) segmentation of complex tasks into logical components given sensor traces of a human performing the task, and (3) measuring HMCS performance. Our goal is to integrate these into a full microsurgical workstation with the ability to automatically “parse” traces of user execution into a task model which is then loaded into the execution environment, providing the user with assistance using online recognition of task state. The major contributions of our work to date include an XML task graph modeling framework and execution engine, an algorithm for real-time segmentation of user actions using continuous Hidden Markov Models, and validation techniques for analyzing the performance of HMCSs.

[1]  P. Fitts.  The information capacity of the human motor system in controlling the amplitude of movement. , 1954, Journal of experimental psychology.

[2]  Louis L. Whitcomb,et al.  Adaptive force control of position/velocity controlled robots: theory and experiment , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[3]  Russell H. Taylor,et al.  A Steady-Hand Robotic System for Microsurgical Augmentation , 1999, Int. J. Robotics Res..

[4]  Gregory D. Hager,et al.  Vision-assisted control for manipulation using virtual fixtures , 2001, IEEE Transactions on Robotics.

[5]  Allison M. Okamura,et al.  Speed-Accuracy Characteristics of Human-Machine Cooperative Manipulation Using Virtual Fixtures With Variable Admittance , 2004, Hum. Factors.

[6]  Gerd Hirzinger,et al.  A Sensor-based Telerobotic System for the Space Robot Experiment ROTEX , 1991, ISER.

[7]  Gregory D. Hager,et al.  Task modeling and specification for modular sensory based human-machine cooperative systems , 2003, IROS.

[8]  Gregory D. Hager,et al.  Building a task language for segmentation and recognition of user input to cooperative manipulation systems , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[9]  Allison M. Okamura,et al.  Effect of virtual fixture compliance on human-machine cooperative manipulation , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Allison M. Okamura,et al.  Recognition of operator motions for real-time assistance using virtual fixtures , 2003, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings..

[11]  Russell H. Taylor,et al.  Simple Biomanipulation Tasks with 'Steady Hand' Cooperative Manipulator , 2003, MICCAI.

[12]  Russell H. Taylor,et al.  Robot-Assisted Stapedotomy: Micropick Fenestration of the Stapes Footplate , 2002, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[13]  Parallel Assembly , 2001, Science.

[14]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[15]  Thomas B. Sheridan,et al.  Human supervisory control of robot systems , 1986, Proceedings. 1986 IEEE International Conference on Robotics and Automation.

[16]  Tzyh Jong Tarn,et al.  Fusion of human and machine intelligence for telerobotic systems , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[17]  Gregory D. Hager,et al.  Spatial motion constraints: theory and demonstrations for robot guidance using virtual fixtures , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[18]  J. Weiss,et al.  Injection of tissue plasminogen activator into a branch retinal vein in eyes with central retinal vein occlusion. , 2001, Ophthalmology.

[19]  Ève Coste-Manière,et al.  Robotic contour following based on visual servoing , 1993, Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '93).

[20]  Gregory D. Hager,et al.  Vision assisted control for manipulation using virtual fixtures: experiments at macro and micro scales , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[21]  John T. Feddema,et al.  Parallel assembly of high-aspect-ratio microstructures , 1999, Optics East.

[22]  Russell H. Taylor,et al.  A Steady-Hand Robotic System for Microsurgical Augmentation , 1999, Int. J. Robotics Res..