Planning for automatic product assembly using reinforcement learning

Abstract Assembly connects functional modules and components of products. The efficient and accurate assembly can improve performance of the product operation and maintenance. It is therefore essential to have an effective method for product assembly. Existing methods of the mechanical product assembly use mainly manual processes that rely on experience of operators. This paper proposes a reinforcement learning method to enable an automatic operation for improved efficiency and accuracy of the mechanical product assembly. A representation of the product assembly is proposed to build a machine learning model. The automatic assembly of product operations is planned by reinforcement learning agents. Constraints of assembly operations are considered to develop searching strategies of the maximum reward for the optimal solution of assembly operations. A quantitative method is proposed to measure efficiency of assembly operations based on the operation time. The proposed method has been applied in the assembly improvement of function modules of an industrial machine.

[1]  S. Mohammad Arabzad,et al.  Efficiency analysis and optimisation of a multi-product assembly line using simulation , 2014 .

[2]  Wenlei Zhang,et al.  Generating interference matrices for automatic assembly sequence planning , 2017 .

[3]  Peihua Gu,et al.  A structure-based approach to evaluation product adaptability in adaptable design , 2011 .

[4]  Hamid Ullah,et al.  Assembly Features: definition, classification, and usefulness in sequence planning , 2009 .

[5]  Stefano Carabelli,et al.  Machine learning and optimization for production rescheduling in Industry 4.0 , 2020, The International Journal of Advanced Manufacturing Technology.

[6]  Sander J. J. Leemans,et al.  Robotic Process Automation: Contemporary themes and challenges , 2020, Comput. Ind..

[7]  Guoliang Liu,et al.  Knowledge-based deep belief network for machining roughness prediction and knowledge discovery , 2020, Comput. Ind..

[8]  Marwan Mattar,et al.  Unity: A General Platform for Intelligent Agents , 2018, ArXiv.

[9]  Soumaya Yacout,et al.  Optimal preventive maintenance policy based on reinforcement learning of a fleet of military trucks , 2019, J. Intell. Manuf..

[10]  BBVL Deepak,et al.  Assembly sequence planning using soft computing methods: A review , 2019 .

[11]  Sara El-Metwally,et al.  Next-Generation Sequence Assembly: Four Stages of Data Processing and Computational Challenges , 2013, PLoS Comput. Biol..

[12]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[13]  F. J. Ramírez,et al.  Disassembly sequence planning: Recent developments and future trends , 2018, Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture.

[14]  Shang Jianzhong,et al.  An efficient method of automatic assembly sequence planning for aerospace industry based on genetic algorithm , 2017 .

[15]  Ronay Ak,et al.  A Survey of the Advancing Use and Development of Machine Learning in Smart Manufacturing. , 2018, Journal of manufacturing systems.

[16]  Peihua Gu,et al.  Assembly sequence planning for open-architecture products , 2018 .

[17]  Micheal Lanham.  Learn Unity ML - Agents - Fundamentals of Unity Machine Learning , 2018 .

[18]  Jungwon Yoon,et al.  Full length Article: Assembly simulations in virtual environments with optimized haptic path and sequence , 2011 .

[19]  Peihua Gu,et al.  Improvement of Product Adaptability by Efficient Module Interactions , 2014 .

[20]  Qi Qi,et al.  Towards automatic visual inspection: A weakly supervised learning method for industrial applicable object detection , 2020, Comput. Ind..

[21]  George Q. Huang,et al.  Dynamic mapping of design elements and affective responses: a machine learning based method for affective design , 2018 .

[22]  Oscar Castillo,et al.  A new approach for dynamic fuzzy logic parameter tuning in Ant Colony Optimization and its application in fuzzy control of a mobile robot , 2015, Appl. Soft Comput..

[23]  Ching-Chin Chern,et al.  Designing a decision-support system for new product sales forecasting , 2010, Expert Syst. Appl..

[24]  Anoop Desai,et al.  Ease of product assembly through a time-based design methodology , 2019, Assembly Automation.

[25]  Jie Zhang,et al.  Framework for the integration of assembly modeling and simulation based on assembly feature pair , 2015 .

[26]  Matthew C. Frank,et al.  Automated Manufacturing Process Selection During Conceptual Design , 2018 .

[27]  Willem F. Bronsvoort,et al.  Assembly features in modeling and planning , 2000 .

[28]  Peihua Gu,et al.  Development of an Open-Architecture Electric Vehicle Using Adaptable Design , 2013 .

[30]  Stefan Lee,et al.  Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[31]  Germanico Gonzalez-Badillo,et al.  The development of a physics and constraint-based haptic virtual assembly system , 2014 .

[32]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[33]  Chulho Chung,et al.  Tool selection-embedded optimal assembly planning in a dynamic manufacturing environment , 2009, Comput. Aided Des..

[34]  Lei Cao,et al.  A study of count-based exploration and bonus for reinforcement learning , 2017, 2017 IEEE 2nd International Conference on Cloud Computing and Big Data Analysis (ICCCBDA).

[35]  Jianhua Liu,et al.  The State-of-the-art, Connotation and Developing Trends of the Products Assembly Technology , 2018 .

[36]  Bart De Schutter,et al.  Residential Demand Response of Thermostatically Controlled Loads Using Batch Reinforcement Learning , 2017, IEEE Transactions on Smart Grid.

[37]  Christiaan J. J. Paredis,et al.  Intelligent Assembly Modeling and Simulation , 2001 .

[38]  Peter Corcoran,et al.  Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning , 2017, ArXiv.

[39]  Brett A. Bryan,et al.  Application note: Parallelization and optimization of spatial analysis for large scale environmental model data assembly , 2012 .

[40]  Yanli Shao,et al.  A machine learning based global simulation data mining approach for efficient design changes , 2018, Adv. Eng. Softw..

[41]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[42]  Zahari Taha,et al.  Development of decision support system for fastener selection in product recovery oriented design , 2014 .

[43]  Emanuele Frontoni,et al.  An Innovative Design Support System for Industry 4.0 Based on Machine Learning Approaches , 2018, 2018 5th International Symposium on Environment-Friendly Energies and Applications (EFEA).

[44]  Dan Guo,et al.  Data-Driven Evolutionary Optimization: An Overview and Case Studies , 2019, IEEE Transactions on Evolutionary Computation.

[45]  Abdelmajid Benamara,et al.  Modelling and implementation of geometric and technological information for disassembly simulation in CAD environment , 2017 .

[46]  Max Welling,et al.  Attention, Learn to Solve Routing Problems! , 2018, ICLR.

[47]  Geoffrey I. Webb,et al.  Encyclopedia of Machine Learning , 2011, Encyclopedia of Machine Learning.

[48]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[49]  Jorge Luis Victória Barbosa,et al.  Machine learning and reasoning for predictive maintenance in Industry 4.0: Current status and challenges , 2020, Comput. Ind..

[50]  G. A. Britton,et al.  Associative assembly design features: concept, implementation and application , 2007 .

[51]  Sun Jin,et al.  Design automation and optimization of assembly sequences for complex mechanical systems , 2010 .

[52]  Akshay Bhadra,et al.  Email Classification Using Supervised Learning Algorithms , 2018 .

[53]  Mobyen Uddin Ahmed,et al.  A Machine Learning Approach to Classify Pedestrians’ Event based on IMU and GPS , 2019 .

[54]  Jing-Rong Li,et al.  A novel connector-knowledge-based approach for disassembly precedence constraint generation , 2010 .

[55]  Hoda A. ElMaraghy,et al.  Knowledge-based model for constructing master assembly sequence , 2015 .

[56]  Xiang Chen,et al.  A flexible assembly retrieval approach for model reuse , 2012, Comput. Aided Des..

[57]  Henry Zhu,et al.  Soft Actor-Critic Algorithms and Applications , 2018, ArXiv.

[58]  S. N. Samy,et al.  Complexity mapping of the product and assembly system , 2012 .

[59]  Peihua Gu,et al.  Accessibility analysis of tools in product module interface operations , 2016 .

[60]  Razvan Pascanu,et al.  Learning to Navigate in Complex Environments , 2016, ICLR.

[61]  Ihsan Sabuncuoglu,et al.  Two exact formulations for disassembly line balancing problems with task precedence diagram construction using an AND/OR graph , 2009 .

[62]  Davy Preuveneers,et al.  The intelligent industry of the future: A survey on emerging trends, research challenges and opportunities in Industry 4.0 , 2017, J. Ambient Intell. Smart Environ..

[63]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[64]  Ian Postlethwaite,et al.  A Probabilistically Robust Path Planning Algorithm for UAVs Using Rapidly-Exploring Random Trees , 2013, J. Intell. Robotic Syst..

[65]  Octavian Morariu,et al.  Machine learning for predictive scheduling and resource allocation in large scale manufacturing systems , 2020, Comput. Ind..

[66]  Peihua Gu,et al.  Interface Adaptability for an Industrial Painting Machine , 2014 .

[67]  Eric Monmasson,et al.  Interests and Limits of Machine Learning-Based Neural Networks for Rotor Position Estimation in EV Traction Drives , 2018, IEEE Transactions on Industrial Informatics.

[68]  Peihua Gu,et al.  Integrated multi-layer representation and ant colony search for product selective disassembly planning , 2016, Comput. Ind..

[69]  Etienne Perot,et al.  Deep Reinforcement Learning framework for Autonomous Driving , 2017, Autonomous Vehicles and Machines.

[70]  Michael I. Jordan,et al.  Machine learning: Trends, perspectives, and prospects , 2015, Science.

[71]  Joshua D. Summers,et al.  Complexity Connectivity Metrics – Predicting Assembly Times with Low Fidelity Assembly CAD Models , 2013 .

[72]  Duhwan Mun,et al.  Deep-learning-based retrieval of piping component catalogs for plant 3D CAD model reconstruction , 2020, Comput. Ind..

[73]  Michael F. Zaeh,et al.  Improving the surface quality of friction stir welds using reinforcement learning and Bayesian optimization , 2020, The International Journal of Advanced Manufacturing Technology.