A retrofit eye gaze tracker for the da Vinci and its integration in task execution using the da Vinci Research Kit

The integration of eye-gaze tracking at the console of surgical robots has the potential to add both speed and functionality to the human-robot interface. In this paper, we present a novel eye gaze tracker that can be integrated as a simple retrofit to the da Vinci console. In particular, the eye tracker can be used with the da Vinci Research Kit (dVRK) to control the patient side manipulators. We present the eye-tracker design and calibration. First, a 2D calibration is carried out to estimate the gaze on the da Vinci's stereoscopic display, followed by a 3D “hand-eye” calibration to estimate the gaze in the surgical scene. Using the dVRK, we demonstrated the use of the eye-tracker to perform a gaze-assisted task, in which the estimated gaze is used as a set point for the robot and the user is guided towards the point of gaze through haptic feedback. The task performed was peg placement and was evaluated in a preliminary user study with five users.

[1]  Guang-Zhong Yang,et al.  Gaze contingent articulated robot control for robot assisted minimally invasive surgery , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Ulrich Büttner,et al.  Smooth pursuit eye movements and optokinetic nystagmus. , 2007, Developments in ophthalmology.

[3]  Donald H. House,et al.  Measuring gaze depth with an eye tracker during stereoscopic display , 2011, APGV '11.

[4]  Clare Kirtley,et al.  The Active Eye: Perspectives on Eye Movement Research , 2014 .

[5]  Craig Hennessey.  Eye-gaze tracking with free head motion , 2005 .

[6]  Donald H. House,et al.  Online 3D Gaze Localization on Stereoscopic Displays , 2014, TAP.

[7]  Moshe Eizenman,et al.  Remote point-of-gaze estimation with single-point personal calibration based on the pupil boundary and corneal reflections , 2011, 2011 24th Canadian Conference on Electrical and Computer Engineering(CCECE).

[8]  Rafael Cabeza,et al.  A Novel Gaze Estimation System With One Calibration Point , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[9]  Guang-Zhong Yang,et al.  Gaze contingent cartesian control of a robotic arm for laparoscopic surgery , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Peter Kazanzides,et al.  An open-source research kit for the da Vinci® Surgical System , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Carlos Hitoshi Morimoto,et al.  Eye gaze tracking techniques for interactive applications , 2005, Comput. Vis. Image Underst..

[12]  Guang-Zhong Yang,et al.  Gaze-Contingent Motor Channelling, haptic constraints and associated cognitive demand for robotic MIS , 2012, Medical Image Anal..

[13]  Guang-Zhong Yang,et al.  Gaze Contingent Depth Recovery and Motion Stabilisation for Minimally Invasive Robotic Surgery , 2004, MIAR.

[14]  Peter Thier,et al.  Neural control of saccadic eye movements. , 2007, Developments in ophthalmology.

[15]  Peter D. Lawrence,et al.  Improving the Accuracy and Reliability of Remote System-Calibration-Free Eye-Gaze Tracking , 2009, IEEE Transactions on Biomedical Engineering.

[16]  Martin Volker Butz,et al.  Goal-oriented gaze strategies afforded by object interaction , 2015, Vision Research.

[17]  Mark R. Wilson,et al.  Gaze training enhances laparoscopic technical skill acquisition and multi-tasking performance: a randomized, controlled study , 2011, Surgical Endoscopy.

[18]  Michael L. Mack,et al.  VISUAL SALIENCY DOES NOT ACCOUNT FOR EYE MOVEMENTS DURING VISUAL SEARCH IN REAL-WORLD SCENES , 2007 .