Simulation-driven machine learning for robotics and automation

Abstract Mass personalization—a megatrend in industrial manufacturing and production—requires fast adaptations of robotics and automation solutions to continually decreasing lot sizes. In this paper, the challenges of applying robot-based automation in a highly individualized production are highlighted. To face these challenges, a framework is proposed that combines latest machine learning (ML) techniques, like deep learning, with high-end physics simulation environments. ML is used for programming and parameterizing machines for a given production task with minimal human intervention. If the simulation environment realistically captures physical properties like forces or elasticity of the real world, it provides a high-quality data source for ML. In doing so, new tasks are mastered in simulation faster than in real-time, while at the same time existing tasks are executed. The functionality of the simulation-driven ML framework is demonstrated on an industrial use case.

[1]  Abhishek Sarkar,et al.  A deep reinforcement learning approach for dynamically stable inverse kinematics of humanoid robots , 2017, 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[2]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Kerstin Küsters.  See, feel, act: Hierarchical learning for complex manipulation skills with multisensory fusion , 2019 .

[4]  Sami Haddadin,et al.  A Framework for Robot Manipulation: Skill Formalism, Meta Learning and Adaptive Control , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[5]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[6]  Xiang Zhang,et al.  Learning Movement Assessment Primitives for Force Interaction Skills , 2018, ArXiv.

[7]  Gianluca Palli,et al.  Integration of Robotic Vision and Tactile Sensing for Wire-Terminal Insertion Tasks , 2019, IEEE Transactions on Automation Science and Engineering.

[8]  Joseph Redmon,et al.  Real-time grasp detection using convolutional neural networks , 2014, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Jörg Franke,et al.  Automatization of the Cable-Routing-Process within the Automated Production of Wiring Systems , 2017 .

[10]  Marco F. Huber,et al.  Large-scale 6D Object Pose Estimation Dataset for Industrial Bin-Picking , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  Sergey Levine,et al.  QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation , 2018, CoRL.

[12]  Lorenzo Molinari Tosatti,et al.  Trajectory-dependent safe distances in human-robot interaction , 2014, Proceedings of the 2014 IEEE Emerging Technology and Factory Automation (ETFA).

[13]  Pieter Abbeel,et al.  Learning Robotic Assembly from CAD , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Sergey Levine,et al.  Residual Reinforcement Learning for Robot Control , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[15]  Bernd Eckstein,et al.  A Framework for Robot Control Software Development and Debugging Using a Real-Time Capable Physics Simulation , 2016 .

[16]  Yuval Tassa,et al.  Simulation tools for model-based robotics: Comparison of Bullet, Havok, MuJoCo, ODE and PhysX , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Sergey Levine,et al.  Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Lihui Wang,et al.  Robotic assembly planning and control with enhanced adaptability through function blocks , 2014, The International Journal of Advanced Manufacturing Technology.

[19]  Andreas Pott,et al.  Intuitive Constraint-Based Robot Programming for Robotic Assembly Tasks* The research leading to these results has received funding from the European Unions Seventh Framework Programme FP7/2013-2017 under grant agreement n 608604 (LIAA: Lean Intelligent Assembly Automation) and Horizon 2020 Research , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Joonho Lee,et al.  Learning agile and dynamic motor skills for legged robots , 2019, Science Robotics.

[21]  Honglak Lee,et al.  Deep learning for detecting robotic grasps , 2013, Int. J. Robotics Res..

[22]  Marcin Andrychowicz,et al.  Hindsight Experience Replay , 2017, NIPS.

[23]  Peter Corke,et al.  Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach , 2018, Robotics: Science and Systems.

[24]  Manuel Fechter,et al.  Integrated risk assessment and safety consideration during design of HRC workplaces , 2017, 2017 22nd IEEE International Conference on Emerging Technologies and Factory Automation (ETFA).

[25]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[26]  S. Jack Hu,et al.  Evolving paradigms of manufacturing: From mass production to mass customization and personalization , 2013 .

[27]  Sergey Levine,et al.  Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection , 2016, Int. J. Robotics Res..

[28]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[29]  Sergey Levine,et al.  Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Hoda A. ElMaraghy,et al.  Smart Adaptable Assembly Systems , 2016 .

[31]  Sergey Levine,et al.  Deep Reinforcement Learning for Vision-Based Robotic Grasping: A Simulated Comparative Evaluation of Off-Policy Methods , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[32]  Christopher Kanan,et al.  Robotic grasp detection using deep convolutional neural networks , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[33]  Pieter Abbeel,et al.  Benchmarking Deep Reinforcement Learning for Continuous Control , 2016, ICML.

[34]  Emmanuel Dellandréa,et al.  Jacquard: A Large Scale Dataset for Robotic Grasp Detection , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[35]  Wojciech Zaremba,et al.  Domain randomization for transferring deep neural networks from simulation to the real world , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[36]  Surya P. N. Singh,et al.  V-REP: A versatile and scalable robot simulation framework , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[37]  Marcin Andrychowicz,et al.  Sim-to-Real Transfer of Robotic Control with Dynamics Randomization , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[38]  Xinyu Liu,et al.  Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics , 2017, Robotics: Science and Systems.