Modulating Human Input for Shared Autonomy in Dynamic Environments

Many robotic tasks require human interaction through teleoperation to achieve high performance. However, in industrial applications these methods often require high levels of concentration and manual dexterity leading to high cognitive loads and dangerous working conditions. Shared autonomy attempts to address these issues by blending human and autonomous reasoning, relieving the burden of precise motor control, tracking, and localization. In this paper we propose an optimization-based representation for shared autonomy in dynamic environments. We ensure real-time tractability by modulating the human input with the information of the changing environment in the same task space, instead of adding it to the optimization cost or constraints. We illustrate the method with two real world applications: grasping objects in a cluttered environment, and a spraying task requiring sprayed linings with greater homogeneity. Finally we use a 7 degree of freedom KUKA LWR arm to simulate the grasping and spraying experiments.

[1]  H. Simon Rational Decision Making in Business Organizations , 1978 .

[2]  O. Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[3]  Louis B. Rosenberg,et al.  Virtual fixtures: Perceptual tools for telerobotic manipulation , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[4]  Marcel Honegger,et al.  Redundancy resolution of a Cartesian space operated heavy industrial manipulator , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[5]  Toshiyuki Inagaki,et al.  Adaptive Automation: Sharing and Trading of Control , 2001 .

[6]  Michael A. Saunders,et al.  SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization , 2002, SIAM J. Optim..

[7]  Thomas B. Sheridan,et al.  Telerobotics, Automation, and Human Supervisory Control , 2003 .

[8]  S. Schaal Dynamic Movement Primitives -A Framework for Motor Control in Humans and Humanoid Robotics , 2006 .

[9]  Stefan Schaal,et al.  Movement reproduction and obstacle avoidance with dynamic movement primitives and potential fields , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[10]  J. Trinkle,et al.  A Local Collision Avoidance Method for Non-strictly Convex Polyhedra , 2009 .

[11]  Emilio Frazzoli,et al.  Anytime Motion Planning using the RRT* , 2011, 2011 IEEE International Conference on Robotics and Automation.

[12]  Howard Jay Chizeck,et al.  Forbidden-region virtual fixtures from streaming point clouds: Remotely touching and protecting a beating heart , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  Aude Billard,et al.  A dynamical system approach to realtime obstacle avoidance , 2012, Autonomous Robots.

[14]  Nicholas Roy,et al.  Hierarchical Motion Planning in Topological Representations , 2012, Robotics: Science and Systems.

[15]  Kris K. Hauser,et al.  Recognition, prediction, and planning for assisted teleoperation of freeform tasks , 2012, Autonomous Robots.

[16]  Siddhartha S. Srinivasa,et al.  A policy-blending formalism for shared control , 2013, Int. J. Robotics Res..

[17]  Rüdiger Dillmann,et al.  Anticipate your surroundings: Predictive collision detection between dynamic obstacles and planned robot trajectories on the GPU , 2015, 2015 European Conference on Mobile Robots (ECMR).

[18]  Yiming Yang,et al.  Real-time motion adaptation using relative distance space representation , 2015, 2015 International Conference on Advanced Robotics (ICAR).

[19]  Vladimir Ivan,et al.  Space-time area coverage control for robot motion synthesis , 2015, 2015 International Conference on Advanced Robotics (ICAR).

[20]  Paolo Robuffo Giordano,et al.  A visual-based shared control architecture for remote telemanipulation , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  Yiming Yang,et al.  Robust shared autonomy for mobile manipulation with continuous scene monitoring , 2017, 2017 13th IEEE Conference on Automation Science and Engineering (CASE).

[22]  Gerhard Neumann,et al.  A learning-based shared control architecture for interactive task execution , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Scott Kuindersma,et al.  Director: A User Interface Designed for Robot Operation with Shared Autonomy , 2017, J. Field Robotics.

[24]  Siddhartha S. Srinivasa,et al.  Human-Robot Mutual Adaptation in Shared Autonomy , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[25]  Siddhartha S. Srinivasa,et al.  Shared autonomy via hindsight optimization for teleoperation and teaming , 2017, Int. J. Robotics Res..

[26]  Bilge Mutlu,et al.  An Autonomous Dynamic Camera Method for Effective Remote Teleoperation , 2018, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[27]  Yiming Yang,et al.  EXOTica: An Extensible Optimization Toolset for Prototyping and Benchmarking Motion Planning and Control , 2018, Studies in Computational Intelligence.

[28]  Sethu Vijayakumar,et al.  Comparing Alternate Modes of Teleoperation for Constrained Tasks , 2019, 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE).