Recognition and Pose Estimation of Auto Parts for an Autonomous Spray Painting Robot

The autonomous operation of industrial robots with minimal human supervision has always been in high demand. To prepare the autonomous operation of a car part spray painting robot, novel object detection, and pose estimation algorithms have been developed in this paper. The object detection part used principal components analysis (PCA) to reduce the dimension of three-dimensional (3-D) point cloud to 2-D binary image. Distance measure between the auto and cross correlation of the binary features was established to find out the similarity between them. Resultantly, the type of auto part was successfully obtained. Furthermore, iterative closest point (ICP) algorithm was used to estimate the pose difference of the auto part with respect to the camera reference frame, which was mounted on the robot. An issue with ICP's lack of robustness to local minimum was solved by the combination of ICP and genetic algorithm (GA). This allowed the optimization of pose error and addressed the problem of local minimum entrapment in ICP. For experimental validation: the proposed object recognition pipeline was implemented in both serial and parallel programming paradigms. The results were obtained for the acquired point clouds of side body car parts and compared with the major 3-D object detection systems in terms of computational cost. Pose estimation error was calculated with both ICP and the modified point set registration schemes, and it was shown to be decreasing in the case of later. All shown results supported the research claims.

[1]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[2]  Markus Vincze,et al.  A Global Hypothesis Verification Framework for 3D Object Recognition in Clutter , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Andriy Myronenko,et al.  Point Set Registration: Coherent Point Drift , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Honghai Liu,et al.  Exploring Human Hand Capabilities Into Embedded Multifingered Object Manipulation , 2011, IEEE Transactions on Industrial Informatics.

[5]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[6]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Gao Hong,et al.  An improved method for registration of point cloud , 2017 .

[8]  Shai Avidan,et al.  Fast-Match: Fast Affine Template Matching , 2013, International Journal of Computer Vision.

[9]  Keiichi Abe,et al.  Topological structural analysis of digitized binary images by border following , 1985, Comput. Vis. Graph. Image Process..

[10]  Ling Shao,et al.  Progressive Shape-Distribution-Encoder for Learning 3D Shape Representation , 2017, IEEE Transactions on Image Processing.

[11]  Dana H. Ballard,et al.  Generalizing the Hough transform to detect arbitrary shapes , 1981, Pattern Recognit..

[12]  Baba C. Vemuri,et al.  Robust Point Set Registration Using Gaussian Mixture Models , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Mario Fernando Montenegro Campos,et al.  Spatial Density Patterns for Efficient Change Detection in 3D Environment for Autonomous Surveillance Robots , 2014, IEEE Transactions on Automation Science and Engineering.

[14]  Benjamin Bustos,et al.  Harris 3D: a robust extension of the Harris operator for interest point detection on 3D meshes , 2011, The Visual Computer.

[15]  Gary R. Bradski,et al.  Fast 3D recognition and pose using the Viewpoint Feature Histogram , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Hua Liu,et al.  Automatic Registration of TLS-TLS and TLS-MLS Point Clouds Using a Genetic Algorithm , 2017, Sensors.

[17]  Alex Zelinsky,et al.  Learning OpenCV---Computer Vision with the OpenCV Library (Bradski, G.R. et al.; 2008)[On the Shelf] , 2009, IEEE Robotics & Automation Magazine.

[18]  Martin Buss,et al.  Fusing laser and vision data with a genetic ICP algorithm , 2008, 2008 10th International Conference on Control, Automation, Robotics and Vision.

[19]  Wenyu Liu,et al.  Fan Shape Model for object detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Zoubin Ghahramani,et al.  Unifying linear dimensionality reduction , 2014, 1406.0873.

[21]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[24]  Marcus Gallagher,et al.  Visualization of learning in multilayer perceptron networks using principal component analysis , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[25]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[26]  Radu Bogdan Rusu,et al.  Semantic 3D Object Maps for Everyday Robot Manipulation , 2013, Springer Tracts in Advanced Robotics.

[27]  Cang Ye,et al.  An Indoor Wayfinding System Based on Geometric Features Aided Graph SLAM for the Visually Impaired , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[28]  Yu Zhong,et al.  Intrinsic shape signatures: A shape descriptor for 3D object recognition , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[29]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[30]  Mohammed Bennamoun,et al.  3D Object Recognition in Cluttered Scenes with Local Surface Features: A Survey , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Allen R. Tannenbaum,et al.  Point Set Registration via Particle Filtering and Stochastic Dynamics , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Markus Vincze,et al.  A Global Hypotheses Verification Method for 3D Object Recognition , 2012, ECCV.

[34]  Gangqi Dong,et al.  Position-based visual servo control of autonomous robotic manipulators , 2015 .

[35]  Tom Drummond,et al.  Faster and Better: A Machine Learning Approach to Corner Detection , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Yue Gao,et al.  Multi-View 3D Object Retrieval With Deep Embedding Network , 2016, IEEE Transactions on Image Processing.

[37]  Huijun Gao,et al.  Automated Inspection of E-Shaped Magnetic Core Elements Using K-tSL-Center Clustering and Active Shape Models , 2013, IEEE Transactions on Industrial Informatics.

[38]  Nassir Navab,et al.  Model globally, match locally: Efficient and robust 3D object recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[39]  Kao-Shing Hwang,et al.  Motion Segmentation and Balancing for a Biped Robot's Imitation Learning , 2017, IEEE Transactions on Industrial Informatics.

[40]  Su-Lin Lee,et al.  A Multirobot Cooperation Framework for Sewing Personalized Stent Grafts , 2017, IEEE Transactions on Industrial Informatics.

[41]  Leonidas J. Guibas,et al.  Volumetric and Multi-view CNNs for Object Classification on 3D Data , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Ko Nishino,et al.  3D Geometric Scale Variability in Range Images: Features and Descriptors , 2012, International Journal of Computer Vision.

[43]  Xiaowu Chen,et al.  3D Mesh Labeling via Deep Convolutional Neural Networks , 2015, ACM Trans. Graph..

[44]  Ming Yang,et al.  Cubature Split Covariance Intersection Filter-Based Point Set Registration , 2018, IEEE Transactions on Image Processing.

[45]  Hongchao Ma,et al.  DEM Retrieval From Airborne LiDAR Point Clouds in Mountain Areas via Deep Neural Networks , 2017, IEEE Geoscience and Remote Sensing Letters.