Bimanual robot control for surface treatment tasks

This work develops a method to perform surface treatment tasks using a bimanual robotic system, i.e. two robot arms cooperatively performing the task. In particular, one robot arm holds the workpiece while the other robot arm has the treatment tool attached to its end-effector. Moreover, the human user teleoperates all the six coordinates of the former robot arm and two coordinates of the latter robot arm, i.e. the teleoperator can move the treatment tool on the plane given by the workpiece surface. Furthermore, a force sensor attached to the treatment tool is used to automatically attain the desired pressure between the tool and the workpiece and to automatically keep the tool orientation orthogonal to the workpiece surface. In addition, to assist the human user during the teleoperation, several constraints are defined for both robot arms in order to avoid exceeding the allowed workspace, e.g. to avoid collisions with other objects in the environment. The theory used in this work to develop the bimanual robot control relies on sliding mode control and task prioritisation. Finally, the feasibility and effectiveness of the method are shown through experimental results using two robot arms.

[1]  Davide Nicolis,et al.  Occlusion-Free Visual Servoing for the Shared Autonomy Teleoperation of Dual-Arm Robots , 2018, IEEE Robotics and Automation Letters.

[3]  Cheng-Chew Lim,et al.  Neural network adaptive dynamic sliding mode formation control of multi-agent systems , 2020, Int. J. Syst. Sci..

[4]  Arie Levant,et al.  Quasi-continuous high-order sliding-mode controllers , 2005, IEEE Transactions on Automatic Control.

[5]  Antonio Sala,et al.  A path conditioning method with trap avoidance , 2012, Robotics Auton. Syst..

[6]  Chunhong Zheng,et al.  A new nonsingular integral terminal sliding mode control for robot manipulators , 2020, Int. J. Syst. Sci..

[7]  Christopher Edwards,et al.  Sliding mode control : theory and applications , 1998 .

[8]  Ameer Hamza Khan,et al.  Sliding Mode Control With PID Sliding Surface for Active Vibration Damping of Pneumatically Actuated Soft Robots , 2020, IEEE Access.

[9]  Matteo Parigi Polverini,et al.  A constraint-based programming approach for robotic assembly skills implementation , 2019, Robotics Comput. Integr. Manuf..

[10]  Sotiris Makris,et al.  Dual arm robot in cooperation with humans for flexible assembly , 2017 .

[12]  Jianwei Zhang,et al.  Coordinated control of a dual-arm robot for surgical instrument sorting tasks , 2019, Robotics Auton. Syst..

[13]  Claudio Pacchierotti,et al.  Haptic-Based Shared-Control Methods for a Dual-Arm System , 2018, IEEE Robotics and Automation Letters.

[14]  Shuai Li,et al.  A Sparse Optimization-Based Control Method for Manipulator With Simultaneous Potential Energy Minimization , 2021, IEEE Transactions on Circuits and Systems II: Express Briefs.

[15]  Aude Billard,et al.  Benchmark for Bimanual Robotic Manipulation of Semi-Deformable Objects , 2020, IEEE Robotics and Automation Letters.

[16]  Bilge Mutlu,et al.  Shared control–based bimanual robot manipulation , 2019, Science Robotics.

[17]  Guangming Shi,et al.  Motor-Imagery-Based Teleoperation of a Dual-Arm Robot Performing Manipulation Tasks , 2019, IEEE Transactions on Cognitive and Developmental Systems.

[18]  Roi Poranne,et al.  A Multi-Level Optimization Framework for Simultaneous Grasping and Motion Planning , 2020, IEEE Robotics and Automation Letters.

[19]  Russell H. Taylor,et al.  Spatial Motion Constraints for Robot Assisted Suturing Using Virtual Fixtures , 2005, MICCAI.

[20]  Arianna Menciassi,et al.  Haptic Intracorporeal Palpation Using a Cable-Driven Parallel Robot: A User Study , 2020, IEEE Transactions on Biomedical Engineering.

[21]  Roemi Fernández,et al.  Robotic Aubergine Harvesting Using Dual-Arm Manipulation , 2020, IEEE Access.

[22]  Min-Cheol Lee,et al.  Trajectory Planning with Collision Avoidance for Redundant Robots Using Jacobian and Artificial Potential Field-based Real-time Inverse Kinematics , 2020 .

[23]  Panfeng Huang,et al.  Mode Switching-Based Symmetric Predictive Control Mechanism for Networked Teleoperation Space Robot System , 2019, IEEE/ASME Transactions on Mechatronics.

[24]  Guillermo Heredia,et al.  Compliant Bimanual Aerial Manipulation: Standard and Long Reach Configurations , 2020, IEEE Access.

[25]  Danica Kragic,et al.  Benchmarking Bimanual Cloth Manipulation , 2020, IEEE Robotics and Automation Letters.

[26]  Luis Gracia,et al.  Trajectory tracking with a 6R serial industrial robot with ordinary and non-ordinary singularities , 2009 .

[27]  Dieter Schmalstieg,et al.  High-Level Teleoperation System for Aerial Exploration of Indoor Environments , 2019, Front. Robot. AI.

[28]  Vadim I. Utkin,et al.  Sliding mode control , 2004 .

[29]  Panfeng Huang,et al.  Predictive Approach for Sensorless Bimanual Teleoperation Under Random Time Delays With Adaptive Fuzzy Control , 2018, IEEE Transactions on Industrial Electronics.

[30]  Xiaofei Zhang,et al.  Adaptive sliding mode control with information concentration estimator for a robot arm , 2020, Int. J. Syst. Sci..

[31]  Loredana Zollo,et al.  Teleoperated control based on virtual fixtures for a redundant surgical system , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Danica Kragic,et al.  Dual arm manipulation - A survey , 2012, Robotics Auton. Syst..

[33]  Yun-Hui Liu,et al.  Dual-Arm Robotic Needle Insertion With Active Tissue Deformation for Autonomous Suturing , 2019, IEEE Robotics and Automation Letters.

[34]  Kostas E. Bekris,et al.  Task-Driven Perception and Manipulation for Constrained Placement of Unknown Objects , 2020, IEEE Robotics and Automation Letters.

[35]  Timothy Bretl,et al.  Motion Planning for Dual-Arm Manipulation of Elastic Rods , 2020, IEEE Robotics and Automation Letters.

[36]  Yiannis Demiris,et al.  Haptic and Visual Feedback Assistance for Dual-Arm Robot Teleoperation in Surface Conditioning Tasks , 2020, IEEE Transactions on Haptics.

[37]  Fanny Ficuciello,et al.  Haptic-guided shared control for needle grasping optimization in minimally invasive robotic surgery , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[38]  Aude Billard,et al.  A unified framework for coordinated multi-arm motion planning , 2018, Int. J. Robotics Res..

[39]  Ameer Hamza Khan,et al.  Tracking control of redundant mobile manipulator: An RNN based metaheuristic approach , 2020, Neurocomputing.

[40]  Alonso H. Vera,et al.  No AI Is an Island: The Case for Teaming Intelligence , 2019, AI Mag..

[41]  Shijie Guo,et al.  Position Adjustment Control of A Nursing-care Robot Holding A Patient in Its Arms , 2019, 2019 IEEE International Conference on Mechatronics and Automation (ICMA).

[42]  Giuseppe Oriolo,et al.  Kinematically Redundant Manipulators , 2008, Springer Handbook of Robotics.

[43]  Edin Omerdic,et al.  Underwater manipulators: A review , 2018, Ocean Engineering.

[44]  Manuel G. Catalano,et al.  On the role of wearable haptics for force feedback in teleimpedance control for dual-arm robotic teleoperation , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[45]  Kensuke Harada,et al.  Integrating combined task and motion planning with compliant control , 2020, International Journal of Intelligent Robotics and Applications.

[46]  S. LaValle,et al.  Motion Planning , 2008, Springer Handbook of Robotics.

[47]  Shuai Li,et al.  Recurrent Neural Networks-Based Collision-Free Motion Planning for Dual Manipulators Under Multiple Constraints , 2020, IEEE Access.

[48]  Gen-Min Lin,et al.  A 12-Lead ECG-Based System With Physiological Parameters and Machine Learning to Identify Right Ventricular Hypertrophy in Young Adults , 2020, IEEE Journal of Translational Engineering in Health and Medicine.

[49]  G. Oriolo,et al.  Robotics: Modelling, Planning and Control , 2008 .

[50]  J. Ernesto Solanes,et al.  Teleoperation of industrial robot manipulators based on augmented reality , 2020, The International Journal of Advanced Manufacturing Technology.

[51]  Hiromitsu Fujii,et al.  Development of Perilous Environment Estimation System Using a Teleoperated Rescue Robot with On-board LiDAR* , 2019, 2019 IEEE/SICE International Symposium on System Integration (SII).

[52]  Jan Rosell,et al.  Motion Planning by Demonstration With Human-Likeness Evaluation for Dual-Arm Robots , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[53]  Claudio Pacchierotti,et al.  A Haptic Shared-Control Architecture for Guided Multi-Target Robotic Grasping , 2020, IEEE Transactions on Haptics.

[54]  Xinwei Cao,et al.  A Fault-Tolerant Method for Motion Planning of Industrial Redundant Manipulator , 2020, IEEE Transactions on Industrial Informatics.

[55]  Dmitry Berenson,et al.  Manipulating deformable objects by interleaving prediction, planning, and control , 2020, Int. J. Robotics Res..

[56]  C. James Taylor,et al.  Vision-Based Assisted Tele-Operation of a Dual-Arm Hydraulically Actuated Robot for Pipe Cutting and Grasping in Nuclear Environments , 2019, Robotics.

[57]  Zhibo Pang,et al.  Teleoperation of Collaborative Robot for Remote Dementia Care in Home Environments , 2020, IEEE Journal of Translational Engineering in Health and Medicine.

[58]  Nathan Elangovan,et al.  An Intuitive, Affordances Oriented Telemanipulation Framework for a Dual Robot Arm Hand System: On the Execution of Bimanual Tasks , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[59]  Andouglas Gonçalves da Silva Silva Junior,et al.  High-Level Path Planning for an Autonomous Sailboat Robot Using Q-Learning , 2020, Sensors.

[60]  T. Yoshikawa,et al.  Task-Priority Based Redundancy Control of Robot Manipulators , 1987 .

[61]  Byung-Ju Yi,et al.  Image-Guided Dual Master–Slave Robotic System for Maxillary Sinus Surgery , 2018, IEEE Transactions on Robotics.

[62]  Aitor Ibarguren,et al.  Dual Arm Co-Manipulation Architecture with Enhanced Human–Robot Communication for Large Part Manipulation , 2020, Sensors.

[63]  Hao-Chi Chang,et al.  Sliding mode control on electro-mechanical systems , 1999 .

[64]  Yu Wang,et al.  Human-like coordination motion learning for a redundant dual-arm robot , 2019, Robotics and Computer-Integrated Manufacturing.

[65]  Gerald Brantner,et al.  Controlling Ocean One: Human–robot collaboration for deep‐sea manipulation , 2020, J. Field Robotics.

[66]  Kai Xu,et al.  Review of surgical robotic systems for keyhole and endoscopic procedures: state of the art and perspectives , 2020, Frontiers of Medicine.

[67]  Arie Levant,et al.  Higher-order sliding modes, differentiation and output-feedback control , 2003 .