Surface-Type Classification Using RGB-D

This paper proposes an approach to improve surface-type classification of images containing inconsistently illuminated surfaces. When a mobile inspection robot is visually inspecting surface-types in a dark environment and a directional light source is used to illuminate the surfaces, the images captured may exhibit illumination variance that can be caused by the orientation and distance of the light source relative to the surfaces. In order to accurately classify the surface-types in these images, either the training image dataset needs to completely incorporate the illumination variance or a way to extract color features that can provide high classification accuracy needs to be identified. In this paper diffused reflectance values are extracted as new color features to classifying surface-types. In this approach, Red, Green, Blue-Depth (RGB-D) data is collected from the environment, and a reflectance model is used to calculate a diffused reflectance value for a pixel in each Red, Green, Blue (RGB) color channel. The diffused reflectance values can be used to train a multiclass support vector machine classifier to classify surface-types. Experiments are conducted in a mock bridge maintenance environment using a portable RGB-Depth sensor package with an attached light source to collect surface-type data. The performance of a classifier trained with diffused reflectance values is compared against classifiers trained with other color features including RGB and L*a*b* color spaces. Results show that the classifier trained with the diffused reflectance values can achieve consistently higher classification accuracy than the classifiers trained with RGB and L*a*b* features. For test images containing a single surface plane, diffused reflectance values consistently provide greater than 90% classification accuracy; and for test images containing a complex scene with multiple surface-types and surface planes, diffused reflectance values are shown to provide an increase in overall accuracy over RGB and L*a*b* by 49.24% and 13.66%, respectively.

[1]  Luh-Maan Chang,et al.  Automated bridge coating defect recognition using adaptive ellipse approach , 2009 .

[2]  Mehran Yazdi,et al.  Steel Surface Defect Detection Using Texture Segmentation Based on Multifractal Dimension , 2009, 2009 International Conference on Digital Image Processing.

[3]  Luis-Manuel Tomás-Balibrea,et al.  Automatic system for quality-based classification of marble textures , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[4]  Luh-Maan Chang,et al.  Support-vector-machine-based method for automated steel bridge rust assessment , 2012 .

[5]  Stefan B. Williams,et al.  Colour-Consistent Structure-from-Motion Models using Underwater Imagery , 2012, Robotics: Science and Systems.

[6]  Sangwook Lee,et al.  Automated recognition of surface defects using digital color image processing , 2006 .

[7]  Andrew Gardner,et al.  A lighting reproduction approach to live-action compositing , 2002, SIGGRAPH.

[8]  Dikai Liu,et al.  Image segmentation for surface material-type classification using 3D geometry information , 2010, The 2010 IEEE International Conference on Information and Automation.

[9]  S. Sathiya Keerthi,et al.  Which Is the Best Multiclass SVM Method? An Empirical Study , 2005, Multiple Classifier Systems.

[10]  Nong Sang,et al.  Robust Illumination Invariant Texture Classification Using Gradient Local Binary Patterns , 2011, 2011 International Workshop on Multi-Platform/Multi-Sensor Remote Sensing and Mapping.

[11]  Atsushi Yamashita,et al.  Color Registration of Underwater Images for Underwater Sensing with Consideration of Light Attenuation , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[12]  T. Higuchi,et al.  Rust classification using image analysis of steel structures , 2008, 2008 3rd International Conference on Sensing Technology.

[13]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2008, Texts in Computer Science.

[14]  Chih-Jen Lin,et al.  Generalized Bradley-Terry Models and Multi-Class Probability Estimates , 2006, J. Mach. Learn. Res..

[15]  Oliver Wang,et al.  Material classification using BRDF slices , 2009, CVPR.

[16]  Junyu Dong,et al.  Three-Dimensional Surface Texture Classification Based on Support Vector Machines and Wavelet Packets , 2008, 2008 Second International Symposium on Intelligent Information Technology Application.

[17]  Radu Bogdan Rusu,et al.  Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments , 2010, KI - Künstliche Intelligenz.

[18]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[19]  Katsushi Ikeuchi,et al.  Light source position and reflectance estimation from a single view without the distant illumination assumption , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Fred Nicolls,et al.  SURFACE CLASSIFICATION VIA BRDF PARAMETERS, USING POPULATION MONTE CARLO FOR MRF PARAMETER ESTIMATION , 2010 .

[21]  Gamini Dissanayake,et al.  A robotic system for steel bridge maintenance: Field testing , 2010 .

[22]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[23]  Gamini Dissanayake,et al.  An effective exploration approach to simultaneous mapping and surface material–type identification of complex three‐dimensional environments , 2009, J. Field Robotics.

[24]  Zhen Zhang,et al.  Auto-classification of insect images based on color histogram and GLCM , 2010, 2010 Seventh International Conference on Fuzzy Systems and Knowledge Discovery.

[25]  Wolfgang Förstner,et al.  Plane Detection in Point Cloud Data , 2010 .