An environment state perception method based on knowledge representation in dual-arm robot assembly tasks

In this paper, an environment state perception method is proposed, in which the visual perception, point cloud process and knowledge representation are combined together to handle the perception problem of robot assembly tasks. Different from regular perception systems, objects in assembly workspace are required to be matched with corresponding models in database which records prior information related about assembly operations. Besides, objects’ special local reference frames are also estimated to fit the task requirements. Once works completed, all the information obtained will be utilized to generate an environment state map, which will be used for bi-manual assembly behaviors automatic generating. In the end, the developed environment state perception scheme is experimentally tested on a dual-arm robot assembly system consist of ABB IRB14000. The simulations and experimental results strongly prove that the proposed approach can achieve good environment perception performance.

[1]  Jeremy A. Marvel,et al.  2011 solutions in perception challenge performance metrics and results , 2012, PerMIS.

[2]  Li Wei,et al.  Adaptive Bayesian recognition and pose estimation of 3D industrial objects with optimal feature selection , 2016, 2016 IEEE International Symposium on Assembly and Manufacturing (ISAM).

[3]  Maria Fox,et al.  PDDL2.1: An Extension to PDDL for Expressing Temporal Planning Domains , 2003, J. Artif. Intell. Res..

[4]  Francisco Simões,et al.  Depth-assisted rectification for real-time object detection and pose estimation , 2015, Machine Vision and Applications.

[5]  Peihua Gu,et al.  Social manufacturing as a sustainable paradigm for mass individualization , 2016 .

[6]  Zoltan-Csaba Marton,et al.  Tutorial: Point Cloud Library: Three-Dimensional Object Recognition and 6 DOF Pose Estimation , 2012, IEEE Robotics & Automation Magazine.

[7]  Philips S. Ogun,et al.  3D vision assisted flexible robotic assembly of machine components , 2015, International Conference on Machine Vision.

[8]  Andreas Birk,et al.  The jacobs robotics approach to object recognition and localization in the context of the ICRA'11 Solutions in Perception Challenge , 2012, 2012 IEEE International Conference on Robotics and Automation.

[9]  Oliver Brock,et al.  Analysis and Observations From the First Amazon Picking Challenge , 2016, IEEE Transactions on Automation Science and Engineering.

[10]  Xavier Lladó,et al.  A Method for 6D Pose Estimation of Free-Form Rigid Objects Using Point Pair Features on Range Data , 2018, Sensors.

[11]  Yasheng Chen,et al.  Model-Based Workpiece Positioning for Robotic Fixtureless Assembly Using Parallel Monocular Vision System , 2013 .

[12]  Dimitris Mourtzis,et al.  The Evolution of Manufacturing Systems: From Craftsmanship to the Era of Customisation , 2014 .

[13]  Kuan-Ting Yu,et al.  Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Christoph Schnörr,et al.  Model-Based Multiple Rigid Object Detection and Registration in Unstructured Range Data , 2011, International Journal of Computer Vision.

[15]  Craig A. Knoblock,et al.  PDDL-the planning domain definition language , 1998 .

[16]  Hesheng Wang,et al.  Monocular visual object-localization using natural corners for assembly tasks , 2016, 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[17]  Gianni Ferretti,et al.  Model based Detection and 3D Localization of Planar Objects for Industrial Setups , 2016, ICINCO.

[18]  Haibin Yu,et al.  Speedup 3-D Texture-Less Object Recognition Against Self-Occlusion for Intelligent Manufacturing , 2019, IEEE Transactions on Cybernetics.

[19]  Masakatsu G. Fujie,et al.  An improved approach for model-based detection and pose estimation of texture-less objects , 2016, 2016 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO).

[20]  Christian A. Mueller,et al.  No More Heavy Lifting: Robotic Solutions to the Container Unloading Problem , 2016, IEEE Robotics & Automation Magazine.

[21]  Xiang Zhang,et al.  Action representations in robotics: A taxonomy and systematic classification , 2018, Int. J. Robotics Res..

[22]  Chyi-Yeu Lin,et al.  6D pose estimation using an improved method based on point pair features , 2018, 2018 4th International Conference on Control, Automation and Robotics (ICCAR).

[23]  Eren Erdal Aksoy,et al.  An Online Vision System for Understanding Complex Assembly Tasks , 2015, VISAPP.

[24]  Le Minh Duc,et al.  Planar Object Recognition For Bin Picking Application , 2018, 2018 5th NAFOSTED Conference on Information and Computer Science (NICS).

[25]  Amar Ramdane-Cherif,et al.  Knowledge Processing Using EKRL for Robotic Applications , 2017, Int. J. Cogn. Informatics Nat. Intell..

[26]  Xun Xu,et al.  Cloud manufacturing for a service-oriented paradigm shift , 2014, 2014 IEEE International Conference on Industrial Engineering and Engineering Management.