Human-Supervised Semi-Autonomous Mobile Manipulators for Safely and Efficiently Executing Machine Tending Tasks

Mobile manipulators can be used for machine tending and material handling tasks in small volume manufacturing applications. These applications usually have semi-structured work environment. The use of a fully autonomous mobile manipulator for such applications can be risky, as an inaccurate model of the workspace may result in damage to expensive equipment. On the other hand, the use of a fully teleoperated mobile manipulator may require a significant amount of operator time. In this paper, a semi-autonomous mobile manipulator is developed for safely and efficiently carrying out machine tending tasks under human supervision. The robot is capable of generating motion plans from the high-level task description and presenting simulation results to the human for approval. The human operator can authorize the robot to execute the automatically generated plan or provide additional input to the planner to refine the plan. If the level of uncertainty in some parts of the workspace model is high, then the human can decide to perform teleoperation to safely execute the task. Our preliminary user trials show that non-expert operators can quickly learn to use the system and perform machine tending tasks.

[1]  Reid G. Simmons,et al.  An autonomous mobile manipulator for assembly tasks , 2010, Auton. Robots.

[2]  Rafael Rey,et al.  Human-robot co-working system for warehouse automation , 2019, 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA).

[3]  Shantanu Thakar,et al.  Towards Remote Teleoperation of a Semi-Autonomous Mobile Manipulator System in Machine Tending Tasks , 2019 .

[4]  Satyandra K. Gupta,et al.  An Alert-Generation Framework for Improving Resiliency in Human-Supervised, Multi-Agent Teams , 2019, ArXiv.

[5]  Shantanu Thakar,et al.  Accounting for Part Pose Estimation Uncertainties during Trajectory Generation for Part Pick-Up Using Mobile Manipulators , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[6]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[7]  C. James Taylor,et al.  Towards a Cooperative Robotic System for Autonomous Pipe Cutting in Nuclear Decommissioning , 2018, 2018 UKACC 12th International Conference on Control (CONTROL).

[8]  Dieter Schmalstieg,et al.  High-Level Teleoperation System for Aerial Exploration of Indoor Environments , 2019, Front. Robot. AI.

[9]  Riccardo Muradore,et al.  An Energy Tank-Based Interactive Control Architecture for Autonomous and Teleoperated Robotic Surgery , 2015, IEEE Transactions on Robotics.

[10]  Jean Scholtz,et al.  Common metrics for human-robot interaction , 2006, HRI '06.

[11]  Satyandra K. Gupta,et al.  Strategies for Speeding Up Manipulator Path Planning to Find High Quality Paths in Cluttered Environments , 2021, J. Comput. Inf. Sci. Eng..

[12]  Axel Krieger,et al.  Experimental evaluation of contact-less hand tracking systems for tele-operation of surgical tasks , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Satyandra K. Gupta,et al.  Context-Dependent Search for Generating Paths for Redundant Manipulators in Cluttered Environments , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[14]  Satyandra K. Gupta,et al.  Task Assignment and Motion Planning for Bi-Manual Mobile Manipulation , 2019, 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE).

[15]  Ross A. Knepper,et al.  Herb 2.0: Lessons Learned From Developing a Mobile Manipulator for the Home , 2012, Proceedings of the IEEE.

[16]  Alma Y. Alanis,et al.  Teleoperation of mobile manipulators with non-holonomic restrictions , 2016, 2016 IEEE 13th International Conference on Networking, Sensing, and Control (ICNSC).

[17]  Daria Trinitatova,et al.  AeroVr: Virtual Reality-based Teleoperation with Tactile Feedback for Aerial Manipulation , 2019, 2019 19th International Conference on Advanced Robotics (ICAR).

[18]  John R. Wagner,et al.  A Mixed-Initiative Haptic Teleoperation Strategy for Mobile Robotic Systems Based on Bidirectional Computational Trust Analysis , 2017, IEEE Transactions on Robotics.

[19]  D. Caleb Rucker,et al.  Analysis and Validation of a Teleoperated Surgical Parallel Continuum Manipulator , 2016, IEEE Robotics and Automation Letters.

[20]  Satyandra K. Gupta,et al.  Generation of Synchronized Configuration Space Trajectories of Multi-Robot Systems , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[21]  Fiora Pirri,et al.  Object-Centered Teleoperation of Mobile Manipulators With Remote Center of Motion Constraint , 2019, IEEE Robotics and Automation Letters.

[22]  Jeremy A. Marvel,et al.  Simulation Based On-Line Evaluation of Singulation Plans to Handle Perception Uncertainty in Robotic Bin Picking , 2017 .

[23]  Shantanu Thakar,et al.  Towards Time-Optimal Trajectory Planning for Pick-and-Transport Operation with a Mobile Manipulator , 2018, 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE).

[24]  Satyandra K. Gupta,et al.  Incorporating Motion Planning Feasibility Considerations during Task-Agent Assignment to Perform Complex Tasks Using Mobile Manipulators , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Torsten Bertram,et al.  Kinodynamic trajectory optimization and control for car-like robots , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[26]  Luca Bascetta,et al.  Parameterized Model Predictive Control of a Nonholonomic Mobile Manipulator: A Terminal Constraint-Free Approach , 2019, 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE).

[27]  Manuel Brucker,et al.  Toward fully autonomous mobile manipulation for industrial environments , 2017 .

[28]  Oliver Brock,et al.  The UMass Mobile Manipulator UMan: An Experimental Platform for Autonomous Mobile Manipulation , 2006 .

[29]  Shahin Sirouspour,et al.  Task Performance Evaluation of Asymmetric Semiautonomous Teleoperation of Mobile Twin-Arm Robotic Manipulators , 2013, IEEE Transactions on Haptics.

[30]  Raffaello D'Andrea,et al.  Guest editorial: A revolution in the warehouse: a retrospective on Kiva Systems and the grand challenges ahead , 2012, IEEE Trans Autom. Sci. Eng..

[31]  François Michaud,et al.  RTAB‐Map as an open‐source lidar and visual simultaneous localization and mapping library for large‐scale and long‐term online operation , 2018, J. Field Robotics.

[32]  M. R. Wrock,et al.  An automatic switching approach to teleoperation of mobile-manipulator systems using virtual fixtures , 2017, Robotica.

[33]  Satyandra K. Gupta,et al.  Trajectory Planning for Manipulators Operating in Confined Workspaces , 2018, 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE).

[34]  John R. Wagner,et al.  Trust-based mixed-initiative teleoperation of mobile robots , 2016, 2016 American Control Conference (ACC).

[35]  Sonia Chernova,et al.  A Comparison of Remote Robot Teleoperation Interfaces for General Object Manipulation , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[36]  Y. Choi,et al.  Efficacy and Safety of Robotic Procedures Performed Using the da Vinci Robotic Surgical System at a Single Institute in Korea: Experience with 10000 Cases , 2018, Yonsei medical journal.

[37]  Antonio Franchi,et al.  Shared planning and control for mobile robots with integral haptic feedback , 2018, Int. J. Robotics Res..