Human-Centered Collaborative Robots With Deep Reinforcement Learning

We present a reinforcement learning based framework for human-centered collaborative systems. The framework is proactive and balances the benefits of timely actions with the risk of taking improper actions by minimizing the total time spent to complete the task. The framework is learned end-to-end in an unsupervised fashion addressing the perception uncertainties and decision making in an integrated manner. The framework is shown to provide more time-efficient coordination between human and robot partners on an example task of packaging compared to alternatives for which perception and decision-making systems are learned independently, using supervised learning. Two important benefits of the proposed approach are that tedious annotation of motion data is avoided, and the learning is performed on-line.

[1]  Aaron F. Bobick,et al.  Probabilistic human action prediction and wait-sensitive planning for responsive human-robot collaboration , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[2]  Yuichiro Yoshikawa,et al.  Robot gains social intelligence through multimodal deep reinforcement learning , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[3]  Danica Kragic,et al.  A sensorimotor reinforcement learning framework for physical Human-Robot Interaction , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[4]  Danica Kragic,et al.  Exploring Temporal Dependencies in Multimodal Referring Expressions with Mixed Reality , 2019, HCI.

[5]  Hema Swetha Koppula,et al.  Anticipatory Planning for Human-Robot Teams , 2014, ISER.

[6]  Danica Kragic,et al.  Imitating by Generating: Deep Generative Models for Imitation of Interactive Tasks , 2019, Frontiers in Robotics and AI.

[7]  Masayoshi Tomizuka,et al.  Towards Efficient Human-Robot Collaboration With Robust Plan Recognition and Trajectory Prediction , 2020, IEEE Robotics and Automation Letters.

[8]  Dmitry Berenson,et al.  Human-robot collaborative manipulation planning using early prediction of human motion , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Il Hong Suh,et al.  Planning of proactive behaviors for human-robot cooperative tasks under uncertainty , 2014, Knowl. Based Syst..

[10]  Peter Stone,et al.  Deep Recurrent Q-Learning for Partially Observable MDPs , 2015, AAAI Fall Symposia.

[11]  Dahua Lin,et al.  Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition , 2018, AAAI.

[12]  Petter Ögren,et al.  A Survey of Behavior Trees in Robotics and AI , 2020, Robotics Auton. Syst..

[13]  Danica Kragic,et al.  Deep predictive policy training using reinforcement learning , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[14]  Darius Burschka,et al.  Predicting human intention in visual observations of hand/object interactions , 2013, 2013 IEEE International Conference on Robotics and Automation.

[15]  Danica Kragic,et al.  Learning to Estimate Pose and Shape of Hand-Held Objects from RGB Images , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[16]  Kazuhiro Kosuge,et al.  Progress and prospects of the human–robot collaboration , 2017, Autonomous Robots.

[17]  Patric Jensfelt,et al.  Adversarial Feature Training for Generalizable Robotic Visuomotor Control , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Weihua Sheng,et al.  Using human motion estimation for human-robot cooperative manipulation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Chonhyon Park,et al.  Intention-Aware Motion Planning Using Learning Based Human Motion Prediction , 2016, Robotics: Science and Systems.

[20]  Bilge Mutlu,et al.  Anticipatory robot control for efficient human-robot collaboration , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[21]  Lihui Wang,et al.  Deep learning-based human motion recognition for predictive context-aware human-robot collaboration , 2018 .

[22]  Ville Kyrki,et al.  Affordance Learning for End-to-End Visuomotor Robot Control , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  Aaron F. Bobick,et al.  Anticipating human actions for collaboration in the presence of task and sensor uncertainty , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Oliver Kroemer,et al.  Probabilistic movement primitives for coordination of multiple human–robot collaborative tasks , 2017, Auton. Robots.

[25]  Robert X. Gao,et al.  Recurrent neural network for motion trajectory prediction in human-robot collaborative assembly , 2020 .

[26]  Christopher Burgess,et al.  beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework , 2016, ICLR 2016.

[27]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[28]  Petter Ögren,et al.  How Behavior Trees Modularize Hybrid Control Systems and Generalize Sequential Behavior Compositions, the Subsumption Architecture, and Decision Trees , 2017, IEEE Transactions on Robotics.

[29]  Julie A. Shah,et al.  Fast target prediction of human reaching motion for cooperative human-robot manipulation tasks using time series classification , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[30]  Paul N. Bennett,et al.  Guidelines for Human-AI Interaction , 2019, CHI.

[31]  Sergey Levine,et al.  Uncertainty-Aware Reinforcement Learning for Collision Avoidance , 2017, ArXiv.