3D Object Recognition Based on Point Clouds in Underwater Environment with Global Descriptors: A Survey

This paper addresses the problem of object recognition from colorless 3D point clouds in underwater environments. It presents a performance comparison of state-of-the-art global descriptors, which are readily available as open source code. The studied methods are intended to assist Autonomous Underwater Vehicles (AUVs) in performing autonomous interventions in underwater Inspection, Maintenance and Repair (IMR) applications. A set of test objects were chosen as being representative of IMR applications whose shape is typically known a priori. As such, CAD models were used to create virtual views of the objects under realistic conditions of added noise and varying resolution. Extensive experiments were conducted from both virtual scans and from real data collected with an AUV equipped with a fast laser sensor developed in our research centre. The underwater testing was conducted from a moving platform, which can create deformations in the perceived shape of the objects. These effects are considerably more difficult to correct than in above-water counterparts, and therefore may affect the performance of the descriptor. Among other conclusions, the testing we conducted illustrated the importance of matching the resolution of the database scans and test scans, as this significantly impacted the performance of all descriptors except one. This paper contributes to the state-of-the-art as being the first work on the comparison and performance evaluation of methods for underwater object recognition. It is also the first effort using comparison of methods for data acquired with a free floating underwater platform.

[1]  Federico Tombari,et al.  Object Recognition in 3D Scenes with Occlusions and Clutter by Hough Voting , 2010, 2010 Fourth Pacific-Rim Symposium on Image and Video Technology.

[2]  Gary R. Bradski,et al.  Fast 3D recognition and pose using the Viewpoint Feature Histogram , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Andrew McCallum,et al.  Efficiently Inducing Features of Conditional Random Fields , 2002, UAI.

[4]  Nico Blodow,et al.  General 3D modelling of novel objects from a single view , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Andrea Torsello,et al.  A Scale Independent Selection Process for 3D Object Recognition in Cluttered Scenes , 2013, International Journal of Computer Vision.

[6]  Pere Ridao,et al.  Underwater 3D Laser Scanners: The Deformation of the Plane , 2017 .

[7]  Nico Blodow,et al.  Autonomous semantic mapping for robots performing everyday manipulation tasks in kitchen environments , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Zhiguo Cao,et al.  Rotational contour signatures for robust local surface description , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[9]  Markus Vincze,et al.  Ensemble of shape functions for 3D object classification , 2011, 2011 IEEE International Conference on Robotics and Biomimetics.

[10]  Steve Vanlanduit,et al.  3D model assisted fully automated scanning laser Doppler vibrometer measurements , 2017 .

[11]  Emili Boronat Roselló,et al.  ROSAPL: towards a heterogeneous multi‐robot system and Human interaction framework , 2014 .

[12]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[13]  Yi Lin,et al.  Automatic Recognition of Pole-Like Objects from Mobile Laser Scanning Point Clouds , 2018, Remote. Sens..

[14]  Jun Zhou,et al.  Automatic apple recognition based on the fusion of color and 3D feature for robotic fruit picking , 2017, Comput. Electron. Agric..

[15]  Pere Ridao,et al.  Semantic SLAM for an AUV using object recognition from point clouds , 2018 .

[16]  José García Rodríguez,et al.  Object Recognition Pipeline: Grasping in Domestic Environments , 2018 .

[17]  Sven Behnke,et al.  Fast Edge-Based Detection and Localization of Transport Boxes and Pallets in RGB-D Images for Mobile Robot Bin Picking , 2016 .

[18]  Mohammed Bennamoun,et al.  Rotational Projection Statistics for 3D Local Surface Description and Object Recognition , 2013, International Journal of Computer Vision.

[19]  Dirk Kraft,et al.  Geometric Edge Description and Classification in Point Cloud Data with Application to 3D Object Recognition , 2015, VISAPP.

[20]  Nico Blodow,et al.  Towards 3D Point cloud based object maps for household environments , 2008, Robotics Auton. Syst..

[21]  Ingrid Bouwer Utne,et al.  Applicability of current remotely operated vehicle standards and guidelines to autonomous subsea IMR operations , 2015 .

[22]  Michael Beetz,et al.  Robots in the kitchen: Exploiting ubiquitous sensing and actuation , 2008, Robotics Auton. Syst..

[23]  Edin Omerdic,et al.  Collision Detection for Underwater ROV Manipulator Systems , 2018, Sensors.

[24]  Adam F. Gobi,et al.  Towards generalized benthic species recognition and quantification using computer vision , 2010, OCEANS'10 IEEE SYDNEY.

[25]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[26]  David Gerónimo Gómez,et al.  Survey of Pedestrian Detection for Advanced Driver Assistance Systems , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Jens Garstka Learning strategies to select point cloud descriptors for large-scale 3-D object classification , 2016 .

[28]  Babak Taati,et al.  Local shape descriptor selection for object recognition in range data , 2011, Comput. Vis. Image Underst..

[29]  Nico Blodow,et al.  CAD-model recognition and 6DOF pose estimation using 3D cues , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[30]  Markus Vincze,et al.  OUR-CVFH - Oriented, Unique and Repeatable Clustered Viewpoint Feature Histogram for Object Recognition and 6DOF Pose Estimation , 2012, DAGM/OAGM Symposium.

[31]  Federico Tombari,et al.  A combined texture-shape descriptor for enhanced 3D feature matching , 2011, 2011 18th IEEE International Conference on Image Processing.

[32]  Nico Blodow,et al.  Close-range scene segmentation and reconstruction of 3D point cloud maps for mobile manipulation in domestic environments , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Jing Huang,et al.  Automatic 3D industrial point cloud modeling and recognition , 2015, 2015 14th IAPR International Conference on Machine Vision Applications (MVA).

[34]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Markus Vincze,et al.  Automation of “ground truth” annotation for multi-view RGB-D object instance recognition datasets , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[36]  José García Rodríguez,et al.  Detecting and Manipulating Objects with a Social Robot: An Ambient Assisted Living Approach , 2017, ROBOT.

[37]  Son-Cheol Yu,et al.  Development of High-Resolution Acoustic Camera based Real-Time Object Recognition System by using Autonomous Underwater Vehicles. , 2006, OCEANS 2006.

[38]  B. Argall,et al.  Estimation of Surface Geometries in Point Clouds for the Manipulation of Novel Household Objects , 2010 .

[39]  Mohammed Bennamoun,et al.  Three-Dimensional Model-Based Object Recognition and Segmentation in Cluttered Scenes , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Kostas Daniilidis,et al.  Single image 3D object detection and pose estimation for grasping , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[41]  Federico Tombari,et al.  Unique shape context for 3d data description , 2010, 3DOR '10.

[42]  Lawson L. S. Wong,et al.  A Vision-Based System for Grasping Novel Objects in Cluttered Environments , 2007, ISRR.

[43]  Luís A. Alexandre 3D Descriptors for Object and Category Recognition: a Comparative Evaluation , 2012 .

[44]  Federico Tombari,et al.  SHOT: Unique signatures of histograms for surface and texture description , 2014, Comput. Vis. Image Underst..

[45]  Michael Beetz,et al.  Detecting and segmenting objects for mobile manipulation , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[46]  Dieter Fox,et al.  A large-scale hierarchical multi-view RGB-D object dataset , 2011, 2011 IEEE International Conference on Robotics and Automation.

[47]  Michael G. Strintzis,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE Snapshots: A Novel Local Surface , 2022 .

[48]  Yael Edan,et al.  CROPS : high tech agricultural robots , 2014 .

[49]  Jing Huang,et al.  Detecting Objects in Scene Point Cloud: A Combinational Approach , 2013, 2013 International Conference on 3D Vision.

[50]  Weigong Zhang,et al.  Traffic sign detection and classification for Advanced Driver Assistant Systems , 2015, 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD).

[51]  Bernt Schiele,et al.  3D object recognition from range images using local feature histograms , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[52]  C. Barat,et al.  A robust visual attention system for detecting manufactured objects in underwater video , 2006, OCEANS 2006.

[53]  Darwin G. Caldwell,et al.  Dexterous Grasping by Manipulability Selection for Mobile Manipulator With Visual Guidance , 2019, IEEE Transactions on Industrial Informatics.

[54]  S. Gentili,et al.  A hierarchical classification system for object recognition in underwater environments , 2002 .

[55]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[56]  Zoltan-Csaba Marton,et al.  Hierarchical object geometric categorization and appearance classification for mobile manipulation , 2010, 2010 10th IEEE-RAS International Conference on Humanoid Robots.

[57]  Meng-Yin Fu,et al.  A survey of traffic sign recognition , 2010, 2010 International Conference on Wavelet Analysis and Pattern Recognition.

[58]  Markus Vincze,et al.  Multimodal cue integration through Hypotheses Verification for RGB-D object recognition and 6DOF pose estimation , 2013, 2013 IEEE International Conference on Robotics and Automation.

[59]  Pawel Markiewicz,et al.  Review of tracking and object detection systems for advanced driver assistance and autonomous driving applications with focus on vulnerable road users sensing , 2017, KKA.

[60]  Ken Tsutsuguchi,et al.  3D object recognition from large-scale point clouds with global descriptor and sliding window , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[61]  Ashok Kumar Patil,et al.  A LiDAR and IMU Integrated Indoor Navigation System for UAVs and Its Application in Real-Time Pipeline Classification , 2017, Sensors.

[62]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[63]  Mikhail Morozov,et al.  Side scan sonar using for underwater cables & pipelines tracking by means of AUV , 2011, 2011 IEEE Symposium on Underwater Technology and Workshop on Scientific Use of Submarine Cables and Related Technologies.

[64]  Z. Moussavi,et al.  An Intelligent Assistive Robotic Manipulator , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[65]  O. Dancea,et al.  A New SVM-Based Architecture for Object Recognition in Color Underwater Images with Classification Refinement by Shape Descriptors , 2006, 2006 IEEE International Conference on Automation, Quality and Testing, Robotics.

[66]  Nico Blodow,et al.  Persistent Point Feature Histograms for 3D Point Clouds , 2008 .

[67]  Mohammed Bennamoun,et al.  3D Object Recognition in Cluttered Scenes with Local Surface Features: A Survey , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[68]  Marc Carreras,et al.  Girona 500 AUV: From Survey to Intervention , 2012, IEEE/ASME Transactions on Mechatronics.

[69]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[70]  Bernard Chazelle,et al.  Matching 3D models with shape distributions , 2001, Proceedings International Conference on Shape Modeling and Applications.

[71]  Mohammed Elmogy,et al.  3D Object Recognition Based on Local and Global Features Using Point Cloud Library , 2015 .

[72]  Luís Seabra Lopes,et al.  An orthographic descriptor for 3D object learning and recognition , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[73]  Mohammed Bennamoun,et al.  A Comprehensive Performance Evaluation of 3D Local Feature Descriptors , 2015, International Journal of Computer Vision.