Human-robot collaboration in sensorless assembly task learning enhanced by uncertainties adaptation via Bayesian Optimization

Abstract Robots are increasingly exploited in production plants. Within the Industry 4.0 paradigm, the robot complements the human’s capabilities, learning new tasks and adapting itself to compensate for uncertainties. With this aim, the presented paper focuses on the investigation of machine learning techniques to make a sensorless robot able to learn and optimize an industrial assembly task. Relying on sensorless Cartesian impedance control, two main contributions are defined: (1) a task-trajectory learning algorithm based on a few human’s demonstrations (exploiting Hidden Markov Model approach), and (2) an autonomous optimization procedure of the task execution (exploiting Bayesian Optimization). To validate the proposed methodology, an assembly task has been selected as a reference application. The task consists of mounting a gear into its square-section shaft on a fixed base to simulate the assembly of a gearbox. A Franka EMIKA Panda manipulator has been used as a test platform, implementing the proposed methodology. The experiments, carried out on a population of 15 subjects, show the effectiveness of the proposed strategy, making the robot able to learn and optimize its behavior to accomplish the assembly task, even in the presence of task uncertainties.

[1]  Sergey Levine,et al.  Residual Reinforcement Learning for Robot Control , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[2]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[3]  Loris Roveda,et al.  Robust state dependent Riccati equation variable impedance control for robotic force-tracking tasks , 2020, International Journal of Intelligent Robotics and Applications.

[4]  N. Vlassis,et al.  Learning in Robotics , 2006 .

[5]  Jing Xu,et al.  Compare Contact Model-based Control and Contact Model-free Learning: A Survey of Robotic Peg-in-hole Assembly Strategies , 2019, ArXiv.

[6]  Matteo Parigi Polverini,et al.  Performance improvement of implicit integral robot force control through constraint-based optimization , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[7]  Carlos Morato,et al.  Instruction Generation for Assembly Operations Performed by Humans , 2012 .

[8]  Alexander Zelinsky,et al.  Programing by Demonstration: Coping with Suboptimal Teaching Actions , 2003 .

[9]  Richard French,et al.  Intelligent sensing for robotic re-manufacturing in aerospace — An industry 4.0 design based prototype , 2017, 2017 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS).

[10]  Claudio Melchiorri,et al.  Trajectory Planning for Automatic Machines and Robots , 2010 .

[11]  Antoine Cully,et al.  Limbo: A Fast and Flexible Library for Bayesian Optimization , 2016, ArXiv.

[12]  Andrea Lockerd Thomaz,et al.  Human-Robot Co-Creativity: Task Transfer on a Spectrum of Similarity , 2017, ICCC.

[13]  Moonhong Baeg,et al.  Compliance-Based Robotic Peg-in-Hole Assembly Strategy Without Force Feedback , 2017, IEEE Transactions on Industrial Electronics.

[14]  W. Wong,et al.  On ψ-Learning , 2003 .

[15]  Sotiris Makris,et al.  On a human-robot collaboration in an assembly cell , 2017, Int. J. Comput. Integr. Manuf..

[16]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[17]  Dario Piga,et al.  Interaction Force Computation Exploiting Environment Stiffness Estimation for Sensorless Robot Applications , 2020, 2020 IEEE International Workshop on Metrology for Industry 4.0 & IoT.

[18]  De Xu,et al.  Automatic precision robot assembly system with microscopic vision and force sensor , 2019 .

[19]  Lorenzo Molinari Tosatti,et al.  Exploiting impedance shaping approaches to overcome force overshoots in delicate interaction tasks , 2016 .

[20]  Stefan Schaal,et al.  Robot Learning From Demonstration , 1997, ICML.

[21]  Jan Peters,et al.  Machine Learning for motor skills in robotics , 2008, Künstliche Intell..

[22]  James Bergstra,et al.  Setting up a Reinforcement Learning Task with a Real-World Robot , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  Ken Chen,et al.  The learning-based optimization algorithm for robotic dual peg-in-hole assembly , 2018, Assembly Automation.

[24]  Silvio Savarese,et al.  Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[25]  Stefan Schaal,et al.  2008 Special Issue: Reinforcement learning of motor skills with policy gradients , 2008 .

[26]  Alberto Finzi,et al.  Kinesthetic teaching and attentional supervision of structured tasks in human–robot interaction , 2019, Auton. Robots.

[27]  Pedro Arezes,et al.  A Brief Overview of the Use of Collaborative Robots in Industry 4.0: Human Role and Safety , 2019, Studies in Systems, Decision and Control.

[28]  Ross A. Knepper,et al.  IkeaBot: An autonomous multi-robot coordinated furniture assembly system , 2013, 2013 IEEE International Conference on Robotics and Automation.

[29]  P. Fettke,et al.  Industry 4.0 , 2014, Bus. Inf. Syst. Eng..

[30]  Stefan Schaal,et al.  From dynamic movement primitives to associative skill memories , 2013, Robotics Auton. Syst..

[31]  Francesco Braghin,et al.  Iterative Learning Procedure With Reinforcement for High-Accuracy Force Tracking in Robotized Tasks , 2018, IEEE Transactions on Industrial Informatics.

[32]  John R. Anderson,et al.  Interactive Task Learning , 2017, IEEE Intelligent Systems.

[33]  Andrew T. Irish,et al.  Trajectory Learning for Robot Programming by Demonstration Using Hidden Markov Model and Dynamic Time Warping , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[34]  Athanasios S. Polydoros,et al.  Survey of Model-Based Reinforcement Learning: Applications on Robotics , 2017, J. Intell. Robotic Syst..

[35]  Marco Forgione,et al.  Robot control parameters auto-tuning in trajectory tracking applications , 2020 .

[36]  Tim Verbelen,et al.  Learning to Grasp from a Single Demonstration , 2018, ArXiv.

[37]  Nolan Wagener,et al.  Learning contact-rich manipulation skills with guided policy search , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[38]  Silvio Savarese,et al.  Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Tasks , 2019, IEEE Transactions on Robotics.

[39]  Siddhartha S. Srinivasa,et al.  Human-robot shared workspace collaboration via hindsight optimization , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[40]  Nando de Freitas,et al.  A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning , 2010, ArXiv.

[41]  Frederic Maire,et al.  Combining learning from demonstration and search algorithm for dynamic goal-directed assembly task planning , 2018, ICRA 2018.

[42]  Carme Torras,et al.  Adapting robot task planning to user preferences: an assistive shoe dressing example , 2019, Auton. Robots.

[43]  C. S. George Lee,et al.  Residue arithmetic VLSI array architecture for manipulator pseudo-inverse Jacobian computation , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[44]  Sergey Levine,et al.  Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[45]  Aude Billard,et al.  Robot learning by demonstration , 2013, Scholarpedia.

[46]  Orhan Arikan,et al.  Short-time Fourier transform: two fundamental properties and an optimal implementation , 2003, IEEE Trans. Signal Process..

[47]  Francesco Braghin,et al.  6D Virtual Sensor for Wrench Estimation in Robotized Interaction Tasks Exploiting Extended Kalman Filter , 2020, Machines.

[48]  Pieter Abbeel,et al.  Learning Robotic Assembly from CAD , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[49]  P. Alam,et al.  H , 1887, High Explosives, Propellants, Pyrotechnics.

[50]  Pooja Gupta,et al.  Robotics and Industry 4.0 , 2019 .

[51]  Aude Billard,et al.  Learning Compliant Manipulation through Kinesthetic and Tactile Human-Robot Interaction , 2014, IEEE Transactions on Haptics.

[52]  G. Fitzgerald,et al.  'I. , 2019, Australian journal of primary health.

[53]  Gorjan Alagic,et al.  #p , 2019, Quantum information & computation.

[54]  P. Alam.  ‘G’ , 2021, Composites Engineering: An A–Z Guide.

[55]  Rouhollah Rahmatizadeh,et al.  Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-to-End Learning from Demonstration , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[56]  Lorenzo Molinari Tosatti,et al.  On robot dynamic model identification through sub-workspace evolved trajectories for optimal torque estimation , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[57]  Alvaro R. De Pierro,et al.  A modified expectation maximization algorithm for penalized likelihood estimation in emission tomography , 1995, IEEE Trans. Medical Imaging.

[58]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[59]  Yu Zhao,et al.  Teach industrial robots peg-hole-insertion by human demonstration , 2016, 2016 IEEE International Conference on Advanced Intelligent Mechatronics (AIM).

[60]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[61]  Alberto Tellaeche,et al.  PIROS: Cooperative, Safe and Reconfigurable Robotic Companion for CNC Pallets Load/Unload Stations , 2020, EuRoC.

[62]  Maja J. Mataric,et al.  Robot life-long task learning from human demonstrations: a Bayesian approach , 2017, Auton. Robots.

[63]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[64]  Bruno Siciliano,et al.  Robot Force Control , 2000 .

[65]  P. Alam.  ‘S’ , 2021, Composites Engineering: An A–Z Guide.

[66]  Alessandro De Luca,et al.  Dynamic Identification of the Franka Emika Panda Robot With Retrieval of Feasible Parameters Using Penalty-Based Optimization , 2019, IEEE Robotics and Automation Letters.

[67]  Jan Peters,et al.  Model learning for robot control: a survey , 2011, Cognitive Processing.

[68]  Loris Roveda.  Adaptive Interaction Controller for Compliant Robot Base Applications , 2019, IEEE Access.

[69]  Matteo Parigi Polverini,et al.  Sensorless and constraint based peg-in-hole task execution with a dual-arm robot , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[70]  Christian Ott,et al.  Bringing Innovative Robotic Technologies from Research Labs to Industrial End-users - The Experience of the European Robotics Challenges , 2020, EuRoC.