Using visuomotor tendencies to increase control performance in teleoperation

Robotic teleoperation is increasingly being used across the world to perform tasks where it is dangerous or impossible for a human to be physically present and complete the task directly themselves. However, robotic teleoperation is difficult due to the disconnect between the operator and the robot caused by the lack of depth in the video feedback presented to the operator. To overcome this problem, this paper presents a novel human-in-the-loop system in which the user is actively involved in the control. In the presented system, the operator's gaze is employed to intuitively indicate the target while their hands are directing the robot towards it. A novel potential field method is used to combine the indicated target with the operator's joystick commands. The control parameters needed to achieve better motion accuracy and user experience are also discussed. Initial validation results show the ability of the system to improve accuracy, execution time, and reduce operator error.

[1]  Christopher C. Pagano,et al.  Using Radial Outflow to Provide Depth Information During Teleoperation , 2009, PRESENCE: Teleoperators and Virtual Environments.

[2]  C. Gielen,et al.  Geometric computations underlying eye-hand coordination: orientations of the two eyes and the head , 2003, Experimental Brain Research.

[3]  John R. Spofford,et al.  Machine-vision-based teleoperation aid , 1991 .

[4]  Martin V. Butz,et al.  Gaze strategies in object identification and manipulation , 2013, CogSci.

[5]  David Drascic,et al.  Skill Acquisition and Task Performance in Teleoperation Using Monoscopic and Stereoscopic Video Remote Viewing , 1991 .

[6]  Thurston L. Brooks,et al.  Operator vision aids for telerobotic assembly and servicing in space , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[7]  Martin Volker Butz,et al.  Goal-oriented gaze strategies afforded by object interaction , 2015, Vision Research.

[8]  Aamir Shahzad,et al.  Mobile Robot Navigation using Gaze Contingent Dynamic Interface , 2010 .

[9]  Irene Yu-Hua Gu,et al.  Chapter 10 (in Part 2): Detecting Landmine Fields from Low-Resolution Aerial Infrared Images , 2011 .

[10]  F. Vitu,et al.  The role of object affordances and center of gravity in eye movements toward isolated daily-life objects. , 2015, Journal of vision.

[11]  Guoyan Zheng,et al.  Medical Imaging and Augmented Reality , 2016, Lecture Notes in Computer Science.

[12]  Daniela Doroftei,et al.  Using robots in hazardous environments: Landmine detection, de-mining and other applications , 2010 .

[13]  Hemin Omer Latif,et al.  Mobile robot teleoperation through eye-gaze (telegaze) , 2010 .

[14]  Kenko Fujii.  Gaze contingent robotic control in minimally invasive surgery , 2014 .

[15]  Hongen Liao,et al.  Medical Imaging and Augmented Reality, 4th International Workshop, MIAR 2008, Tokyo, Japan, August 1-2, 2008, Proceedings , 2008, MIAR.

[16]  Abhilash Pandya,et al.  A Review of Camera Viewpoint Automation in Robotic and Laparoscopic Surgery , 2014, Robotics.

[17]  J. Pelz,et al.  Oculomotor behavior and perceptual strategies in complex tasks , 2001, Vision Research.

[18]  Rodrigo Ventura,et al.  Immersive 3-D teleoperation of a search and rescue robot using a head-mounted display , 2009, 2009 IEEE Conference on Emerging Technologies & Factory Automation.

[19]  Yu Wang,et al.  Human-Robot Interaction Based on Gaze Gestures for the Drone Teleoperation , 2014 .

[20]  M. Hayhoe,et al.  Look-ahead fixations: anticipatory eye movements in natural tasks , 2007, Experimental Brain Research.

[21]  J D Crawford,et al.  Spatial transformations for eye-hand coordination. , 2004, Journal of neurophysiology.

[22]  Etienne Colle,et al.  Contribution of neuroscience to the teleoperation of rehabilitation robot , 2002, IEEE International Conference on Systems, Man and Cybernetics.

[23]  H. S. Osborne,et al.  The international electrotechnical commission , 1953, Electrical Engineering.

[24]  P. Hoppenot,et al.  Remote control of a biomimetics robot assistance system for disabled persons , 2022 .

[25]  Songpo Li,et al.  Attention-Aware Robotic Laparoscope Based on Fuzzy Interpretation of Eye-Gaze Patterns , 2015 .

[26]  Susanna Nilsson,et al.  Hands Free Interaction with Virtual Information in a Real Environment , 2007 .

[27]  M. Hayhoe,et al.  In what ways do eye movements contribute to everyday activities? , 2001, Vision Research.

[28]  Benoît Herman,et al.  Toward remote teleoperation with eye and hand: A first experimental study , 2013 .

[29]  Guang-Zhong Yang,et al.  Gaze-Contingent Motor Channelling and Haptic Constraints for Minimally Invasive Robotic Surgery , 2008, MICCAI.