A Survey of Environment-, Operator-, and Task-adapted Controllers for Teleoperation Systems

Bilateral haptic teleoperation systems allow humans to perform complex tasks in a remote or inaccessible environment, while providing haptic feedback to the human operator. The incorporation of online gained environment-, operator-, or task-specific (EOT) information in the controller structure can lead to significant improvements in robustness, task performance, feeling of presence, or fidelity without compromising stability. This article provides a classification as well as a survey of approaches, called EOT-adapted controllers, which have been developed in this area. A discussion of improvements and requirements is provided for each method. The performed analysis indicates that several methods require the usage of additional sensors or are based on accurate model assumptions. The benefit of EOT-adapted controllers is mostly application-dependent, as each method focuses on the improvement of a specific aspect like coping with time delay or avoiding forbidden regions.

[1]  Tatsuya Suzuki,et al.  Behavior modeling in man-machine cooperative system based on stochastic switched dynamics , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[2]  Gregory D. Hager,et al.  Dynamic Guidance with Pseudoadmittance Virtual Fixtures , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[3]  Jong Hyeon Park,et al.  Impedance Control with Variable Damping for Bilateral Teleoperation under Time Delay , 2005 .

[4]  Rajiv V. Dubey,et al.  Human-machine cooperative telerobotics using uncertain sensor or model data , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[5]  Masaru Uchiyama,et al.  Model-based space robot teleoperation of ETS-VII manipulator , 2004, IEEE Transactions on Robotics and Automation.

[6]  Woodrow Barfield,et al.  A Review of Presence and Performance in Virtual Environments , 2000, Int. J. Hum. Comput. Interact..

[7]  Blake Hannaford,et al.  Performance evaluation of a six-axis generalized force-reflecting teleoperator , 1991, IEEE Trans. Syst. Man Cybern..

[8]  Rajiv V. Dubey,et al.  Teleoperation assistance through variable velocity mapping , 2001, IEEE Trans. Robotics Autom..

[9]  Blake Hannaford,et al.  Experimental and simulation studies of hard contact in force reflecting teleoperation , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[10]  Won S. Kim,et al.  Demonstration of a high-fidelity predictive/preview display technique for telerobotic servicing in space , 1993, IEEE Trans. Robotics Autom..

[11]  Danica Kragic,et al.  Adaptive Virtual Fixtures for Machine-Assisted Teleoperation Tasks , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[12]  Dale A. Lawrence Stability and transparency in bilateral teleoperation , 1993, IEEE Trans. Robotics Autom..

[13]  Toru Tsumugiwa,et al.  Variable Impedance Control Based on Estimation of Human Arm Stiffness for Human-Robot Cooperative Task : Human-Robot Cooperative Calligraphic task , 2002 .

[14]  M. Zefran,et al.  A notion of passivity for hybrid systems , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[15]  Martin Buss,et al.  Effects of Varied Human Movement Control on Task Performance and Feeling of Telepresence , 2008, EuroHaptics.

[16]  N.V. Thakor,et al.  Adaptive cancelling of physiological tremor for improved precision in microsurgery , 1998, IEEE Transactions on Biomedical Engineering.

[17]  Martin Buss,et al.  Study on Computer Assistance for Telepresent Reaching Movements , 2008, EuroHaptics.

[18]  Robert B. Welch,et al.  How Can We Determine if the Sense of Presence Affects Task Performance? , 1999, Presence.

[19]  Katherine J. Kuchenbecker,et al.  Canceling Induced Master Motion in Force-Reflecting Teleoperation , 2004 .

[20]  Thomas B. Sheridan,et al.  Telerobotics, Automation, and Human Supervisory Control , 2003 .

[21]  Bruce A. Francis,et al.  Bilateral controller for teleoperators with time delay via μ-synthesis , 1995, IEEE Trans. Robotics Autom..

[22]  Yuan F. Zheng,et al.  Intelligent compliant motion control , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[23]  Homayoun Seraji,et al.  Force Tracking in Impedance Control , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[24]  Robert D. Howe,et al.  Virtual Fixtures for Robotic Cardiac Surgery , 2001, MICCAI.

[25]  Stefano Stramigioli,et al.  Contact impedance estimation for robotic systems , 2005, IEEE Trans. Robotics.

[26]  Neville Hogan,et al.  Stable execution of contact tasks using impedance control , 1987, Proceedings. 1987 IEEE International Conference on Robotics and Automation.

[27]  Redwan Alqasemi,et al.  Telemanipulation Assistance Based on Motion Intention Recognition , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[28]  David B. Kaber,et al.  Presence, workload and performance effects of synthetic environment design factors , 2006, Int. J. Hum. Comput. Stud..

[29]  David B. Kaber,et al.  Effects of Visual Interface Design, and Control Mode and Latency on Performance, Telepresence and Workload in a Teleoperation Task , 2000 .

[30]  Katherine J. Kuchenbecker,et al.  Modeling Induced Master Motion in Force-Reflecting Teleoperation , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[31]  Paul Evrard,et al.  Homotopy switching model for dyad haptic interaction in physical collaborative tasks , 2009, World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[32]  Tsuneo Yoshikawa,et al.  Bilateral control of master-slave manipulators for ideal kinesthetic coupling-formulation and experiment , 1994, IEEE Trans. Robotics Autom..

[33]  Tatsuya Suzuki,et al.  Modeling of Human Behavior in Man-Machine Cooperative System Based on Hybrid System Framework , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[34]  Hendrik Van Brussel,et al.  Human-inspired robot assistant for fast point-to-point movements , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[35]  Dan O. Popa,et al.  An analysis of some fundamental problems in adaptive control of force and impedance behavior: theory and experiments , 1995, IEEE Trans. Robotics Autom..

[36]  Yoshiyuki Tanaka,et al.  Analysis and Modeling of Human Impedance Properties for Designing a Human-Machine Control System , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[37]  T. Flash,et al.  The coordination of arm movements: an experimentally confirmed mathematical model , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[38]  Neville Hogan,et al.  Impedance Control: An Approach to Manipulation , 1984, 1984 American Control Conference.

[39]  Michael F. Zäh,et al.  Prediction-based methods for teleoperation across delayed networks , 2007, Multimedia Systems.

[40]  Paul Evrard,et al.  Homotopy-based controller for physical human-robot interaction , 2009, RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication.

[41]  Tamio Arai,et al.  Human-robot cooperative manipulation with motion estimation , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[42]  Martin Buss,et al.  Improvement of model-mediated teleoperation using a new hybrid environment estimation technique , 2010, 2010 IEEE International Conference on Robotics and Automation.

[43]  Ryojun Ikeura,et al.  Investigating the impedance characteristic of human arm for development of robots to co-operate with human operators , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[44]  B. Yalcin,et al.  Environmental Impedance Estimation and Imitation in Haptics by Sliding Mode Neural Networks , 2006, IECON 2006 - 32nd Annual Conference on IEEE Industrial Electronics.

[45]  Petros A. Ioannou,et al.  Adaptive control tutorial , 2006, Advances in design and control.

[46]  Wayne J. Book,et al.  Force reflecting teleoperation with adaptive impedance control , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[47]  Rajiv V. Dubey,et al.  Variable position mapping based assistance in teleoperation for nuclear cleanup , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[48]  Septimiu E. Salcudean,et al.  Analysis of Control Architectures for Teleoperation Systems with Impedance/Admittance Master and Slave Manipulators , 2001, Int. J. Robotics Res..

[49]  Heng Wang,et al.  Reference Trajectory Generation for Force Tracking Impedance Control by Using Neural Network-based Environment Estimation , 2006, 2006 IEEE Conference on Robotics, Automation and Mechatronics.

[50]  Abderrahmane Kheddar,et al.  The hidden robot concept-high level abstraction teleoperation , 1997, Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems. Innovative Robotics for Real-World Applications. IROS '97.

[51]  Blake Hannaford,et al.  Stable haptic interaction with virtual environments , 1999, IEEE Trans. Robotics Autom..

[52]  Toru Tsumugiwa,et al.  Variable impedance control based on estimation of human arm stiffness for human-robot cooperative calligraphic task , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[53]  Josep Amat,et al.  Augmented reality to assist teleoperation working with reduced visual conditions , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[54]  Steven B. Skaar,et al.  Techniques For Collision Prevention, Impact Stability, And Force Control By Space Manipulators , 1994 .

[55]  Yasuyoshi Yokokohji,et al.  Robust variable-scale bilateral control for micro teleoperation , 2008, 2008 IEEE International Conference on Robotics and Automation.

[56]  Gregory D. Hager,et al.  Vision-assisted control for manipulation using virtual fixtures , 2001, IEEE Transactions on Robotics.

[57]  A. Morse,et al.  Basic problems in stability and design of switched systems , 1999 .

[58]  B. Hannaford,et al.  Hidden Markov Model of Force Torque Information in Telemanipulation , 1991 .

[59]  Louis B. Rosenberg,et al.  Virtual fixtures: Perceptual tools for telerobotic manipulation , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[60]  Mark W. Spong,et al.  Bilateral teleoperation: An historical survey , 2006, Autom..

[61]  R. W. Daniel,et al.  Stabilizing Impacts in Force-Reflecting Teleoperation Using Distance-to-Impact Estimates , 2000, Int. J. Robotics Res..

[62]  Kirsten Rassmus-Gröhn,et al.  Supporting presence in collaborative environments by haptic force feedback , 2000, TCHI.

[63]  Hikaru Inooka,et al.  Variable impedance control of a robot for cooperation with a human , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[64]  M. Branicky Stability of switched and hybrid systems , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[65]  Yushing Cheung,et al.  Adaptive force reflecting teleoperation with local intelligence , 2007, Ind. Robot.

[66]  Cristian Secchi,et al.  A Force Dependent Scaling for Improving the Human Perception in Bilateral Teleoperation , 2008, ICRA 2008.

[67]  Nak Young Chong,et al.  Exploring interactive simulator in collaborative multi-site teleoperation , 2001, Proceedings 10th IEEE International Workshop on Robot and Human Interactive Communication. ROMAN 2001 (Cat. No.01TH8591).

[68]  David W. L. Wang,et al.  A Gain-Switching Control Scheme for Position-Error-Based Bilateral Teleoperation: Contact Stability Analysis and Controller Design , 2004, Int. J. Robotics Res..

[69]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[70]  Allison M. Okamura,et al.  Haptic Virtual Fixtures for Robot-Assisted Manipulation , 2005, ISRR.

[71]  Katherine J. Kuchenbecker,et al.  Induced Master Motion in Force-Reflecting Teleoperation , 2006 .

[72]  J. Edward Colgate,et al.  Factors affecting the Z-Width of a haptic display , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[73]  Allison M. Okamura,et al.  Recognition of operator motions for real-time assistance using virtual fixtures , 2003, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings..

[74]  Henrik I. Christensen,et al.  A minimum jerk predictor for teleoperation with variable time delay , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[75]  Kyle B. Reed,et al.  Physical Collaboration of Human-Human and Human-Robot Teams , 2008, IEEE Transactions on Haptics.

[76]  Clément Gosselin,et al.  Safe, Stable and Intuitive Control for Physical Human-Robot Interaction , 2009, 2009 IEEE International Conference on Robotics and Automation.

[77]  Jean-Jacques E. Slotine,et al.  Stable Adaptive Teleoperation , 1990, 1990 American Control Conference.

[78]  Li Huijun,et al.  Virtual-Environment Modeling and Correction for Force-Reflecting Teleoperation With Time Delay , 2007 .

[79]  O. Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[80]  Keyvan Hashtrudi-Zaad,et al.  Predictive Teleoperation using Laser Rangefinder , 2006, 2006 Canadian Conference on Electrical and Computer Engineering.

[81]  Martin Buss,et al.  Robust stability analysis of a bilateral teleoperation system using the parameter space approach , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[82]  Daniel E. Whitney,et al.  Historical Perspective and State of the Art in Robot Force Control , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[83]  Tariq Rahman,et al.  Customized optimal filter for eliminating operator's tremor , 1995, Other Conferences.

[84]  Brenan J. McCarragher,et al.  Human integration into robot control utilising potential fields , 1997, Proceedings of International Conference on Robotics and Automation.

[85]  Gregory D. Hager,et al.  Human-Machine Collaborative Systems for Microsurgical Applications , 2005, Int. J. Robotics Res..

[86]  Allison M. Okamura,et al.  Speed-Accuracy Characteristics of Human-Machine Cooperative Manipulation Using Virtual Fixtures With Variable Admittance , 2004, Hum. Factors.

[87]  Georg Passig,et al.  A Photorealistic Predictive Display , 2004, Presence: Teleoperators & Virtual Environments.

[88]  Alana Sherman,et al.  Bilateral controller design for telemanipulation in soft environments , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[89]  Rajiv V. Dubey,et al.  Model-based variable position mapping for telerobotic assistance in a cylindrical environment , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[90]  Costas S. Tzafestas,et al.  Adaptive impedance control in haptic teleoperation to improve transparency under time-delay , 2008, 2008 IEEE International Conference on Robotics and Automation.

[91]  Inna Sharf,et al.  Contact Stiffness and Damping Estimation for Robotic Systems , 2003, Int. J. Robotics Res..

[92]  Allison M. Okamura,et al.  Effect of virtual fixture compliance on human-machine cooperative manipulation , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[93]  Martin Buss,et al.  An HMM approach to realistic haptic human-robot interaction , 2009, World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[94]  Wayne J. Book,et al.  Environment estimation for enhanced impedance control , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[95]  Clément Gosselin,et al.  General Model of Human-Robot Cooperation Using a Novel Velocity Based Variable Impedance Control , 2007, Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC'07).

[96]  M. Zefran,et al.  Stable haptic interaction with switched virtual environments , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[97]  Kunikatsu Takase,et al.  Multioperator Teleoperation of Multirobot Systems with Time Delay: Part IAids for Collision-Free Control , 2002, Presence: Teleoperators & Virtual Environments.

[98]  Neville Hogan,et al.  Impedance control - An approach to manipulation. I - Theory. II - Implementation. III - Applications , 1985 .

[99]  Fumio Miyazaki,et al.  A simple control design for human-robot coordination based on the knowledge of dynamical role division , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[100]  Probal Mitra,et al.  Model-mediated Telemanipulation , 2008, Int. J. Robotics Res..

[101]  Guillaume Morel,et al.  How can human motion prediction increase transparency? , 2008, 2008 IEEE International Conference on Robotics and Automation.

[102]  Neville Hogan,et al.  Impedance Control: An Approach to Manipulation: Part I—Theory , 1985 .

[103]  Martin Buss,et al.  Position and force augmentation in a telepresence system and their effects on perceived realism , 2009, World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[104]  Mel Slater,et al.  Immersion, presence and performance in virtual environments: an experiment with tri-dimensional chess , 1996, VRST.

[105]  Won S. Kim,et al.  The phantom robot: predictive displays for teleoperation with time delay , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[106]  Blake Hannaford,et al.  Hidden Markov Model Analysis of Force/Torque Information in Telemanipulation , 1991, Int. J. Robotics Res..