Caring About the Human Operator: Haptic Shared Control for Enhanced User Comfort in Robotic Telemanipulation

Haptic shared control enables a human operator and an autonomous controller to share the control of a robotic system using haptic active constraints. It has been used in robotic teleoperation for different purposes, such as navigating along paths minimizing the torques requested to the manipulator or avoiding possibly dangerous areas of the workspace. However, few works have focused on using these ideas to account for the user's comfort. In this article, we present an innovative haptic-enabled shared control approach aimed at minimizing the user's workload during a teleoperated manipulation task. Using an inverse kinematic model of the human arm and the rapid upper limb assessment (RULA) metric, the proposed approach estimates the current user's comfort online. From this measure and an a priori knowledge of the task, we then generate dynamic active constraints guiding the users towards a successful completion of the task, along directions that improve their posture and increase their comfort. Studies with human subjects show the effectiveness of the proposed approach, yielding a 30% perceived reduction of the workload with respect to using standard guided human-in-the-loop teleoperation.

[1]  Brian L. Davies,et al.  Active Constraints/Virtual Fixtures: A Survey , 2014, IEEE Transactions on Robotics.

[2]  B. Argall,et al.  Human-in-the-Loop Optimization of Shared Autonomy in Assistive Robotics , 2017, IEEE Robotics and Automation Letters.

[3]  Claudio Pacchierotti,et al.  A Haptic Shared-Control Architecture for Guided Multi-Target Robotic Grasping , 2020, IEEE Transactions on Haptics.

[4]  Robert Riener,et al.  The Effect of Haptic Guidance on Learning a Hybrid Rhythmic-Discrete Motor Task , 2015, IEEE Transactions on Haptics.

[5]  Nikolaos G. Tsagarakis,et al.  Teleoperation in cluttered environments using wearable haptic feedback , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Claudio Pacchierotti,et al.  Haptic Shared-Control Methods for Robotic Cutting under Nonholonomic Constraints , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[7]  Mark Mulder,et al.  Sharing Control With Haptics , 2012, Hum. Factors.

[8]  Seth Hutchinson,et al.  Visual Servo Control Part I: Basic Approaches , 2006 .

[9]  Claudio Pacchierotti,et al.  Displaying Sensed Tactile Cues with a Fingertip Haptic Device , 2015, IEEE Transactions on Haptics.

[10]  D. Prattichizzo,et al.  Experimental evaluation of magnified haptic feedback for robot‐assisted needle insertion and palpation , 2017, The international journal of medical robotics + computer assisted surgery : MRCAS.

[11]  Keyvan Hashtrudi-Zaad,et al.  Dual-User Teleoperation Systems: New Multilateral Shared Control Architecture and Kinesthetic Performance Measures , 2012, IEEE/ASME Transactions on Mechatronics.

[12]  Angelika Peer,et al.  Exploring the Design Space of Haptic Assistants: The Assistance Policy Module , 2013, IEEE Transactions on Haptics.

[13]  Clément Gosselin,et al.  General Model of Human-Robot Cooperation Using a Novel Velocity Based Variable Impedance Control , 2007, Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC'07).

[14]  Mark Mulder,et al.  Haptic shared control: smoothly shifting control authority? , 2012, Cognition, Technology & Work.

[15]  Kazuo Tanie,et al.  Human-Robot Cooperative Manipulation Using a Virtual Nonholonomic Constraint , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[16]  Paolo Robuffo Giordano,et al.  Human-in-the-loop optimisation: Mixed initiative grasping for optimally facilitating post-grasp manipulative actions , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  François Chaumette,et al.  Visual servo control. I. Basic approaches , 2006, IEEE Robotics & Automation Magazine.

[18]  Mehmet Remzi Dogar,et al.  Planning for Muscular and Peripersonal-Space Comfort During Human-Robot Forceful Collaboration , 2018, 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids).

[19]  Martin Buss,et al.  Towards real-time haptic assistance adaptation optimizing task performance and human effort , 2011, 2011 IEEE World Haptics Conference.

[20]  Michael Damsgaard,et al.  Optimizing Contextual Ergonomics Models in Human-Robot Interaction , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  Donghyuk Kim,et al.  TORM: Collision-Free Trajectory Optimization of Redundant Manipulator given an End-Effector Path , 2019, ArXiv.

[22]  Antonis A. Argyros,et al.  Markerless 3D Human Pose Estimation and Tracking based on RGBD Cameras: an Experimental Evaluation , 2017, PETRA.

[23]  Manuel Lopes,et al.  Postural optimization for an ergonomic human-robot interaction , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[24]  Arash Ajoudani,et al.  Towards ergonomie control of human-robot co-manipulation and handover , 2017, 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids).

[25]  Ronald L. Boring,et al.  Shared understanding for collaborative control , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[26]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[27]  Marcia Kilchenman O'Malley,et al.  Negative efficacy of fixed gain error reducing shared control for training in virtual environments , 2009, TAP.

[28]  Claudio Pacchierotti,et al.  Haptic-Based Shared-Control Methods for a Dual-Arm System , 2018, IEEE Robotics and Automation Letters.

[29]  Jerzy W. Rozenblit,et al.  A haptic guidance system for Computer-Assisted Surgical Training using virtual fixtures , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[30]  L McAtamney,et al.  RULA: a survey method for the investigation of work-related upper limb disorders. , 1993, Applied ergonomics.

[31]  Claudio Pacchierotti,et al.  Design and Evaluation of a Wearable Skin Stretch Device for Haptic Guidance , 2018, IEEE Robotics and Automation Letters.

[32]  Claudio Pacchierotti,et al.  Steering and Control of Miniaturized Untethered Soft Magnetic Grippers With Haptic Assistance , 2018, IEEE Transactions on Automation Science and Engineering.

[33]  E V Biryukova,et al.  Kinematics of human arm reconstructed from spatial tracking system recordings. , 2000, Journal of biomechanics.

[34]  Kosei Kitagaki,et al.  A Practical Redundancy Resolution for 7 DOF Redundant Manipulators with Joint Limits , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[35]  Claudio Pacchierotti,et al.  Cutaneous Haptic Feedback in Robotic Teleoperation , 2015, Springer Series on Touch and Haptic Systems.

[36]  Claudio Pacchierotti,et al.  Experimental evaluation of co‐manipulated ultrasound‐guided flexible needle steering , 2016, The international journal of medical robotics + computer assisted surgery : MRCAS.

[37]  Homayoun Seraji,et al.  Configuration control of redundant manipulators: theory and implementation , 1989, IEEE Trans. Robotics Autom..

[38]  Vincent Hayward,et al.  Wearable Haptic Systems for the Fingertip and the Hand: Taxonomy, Review, and Perspectives , 2017, IEEE Transactions on Haptics.