RGBD Camera Based Material Recognition via Surface Roughness Estimation

Real world objects can be characterized effectively by their shape, color and material types. Material recognition of an arbitrary object at a distance is an important task for the improvement of object recognition, scene understanding, realistic rendering and various virtual and augmented reality applications. Researchers have tried to recognize material types based on color features, however material type of an object is not completely correlated with its visual appearance. In this paper, we propose a simple but effective surface roughness estimation method using single time-offlight (ToF) camera. A set of features extracted from the estimated roughness together with conventional color features are used for material type recognition. Experimental results on our material data set with 122 subjects show promising material type recognition results.

[1]  Iasonas Kokkinos,et al.  Describing Textures in the Wild , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  D. Falie,et al.  Measurements with ToF Cameras and Their Necessary Corrections , 2007, 2007 International Symposium on Signals, Circuits and Systems.

[3]  Edward H. Adelson,et al.  Connecting Look and Feel: Associating the Visual and Tactile Properties of Physical Materials , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[5]  Noah Snavely,et al.  Material recognition in the wild with the Materials in Context Database , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Xiaofeng Ren,et al.  Toward Robust Material Recognition for Everyday Objects , 2011, BMVC.

[7]  Matti Pietikäinen,et al.  LOAD: Local orientation adaptive descriptor for texture and material classification , 2015, Neurocomputing.

[8]  Edward H. Adelson,et al.  Exploring features in a Bayesian framework for material recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[10]  Gregory J. Ward,et al.  Measuring and modeling anisotropic reflection , 1992, SIGGRAPH.

[11]  Alexei A. Efros,et al.  A 4D Light-Field Dataset and CNN Architectures for Material Recognition , 2016, ECCV.

[12]  Luc Van Gool,et al.  A Training-free Classification Framework for Textures, Writers, and Materials , 2012, BMVC.

[13]  Rong Xiao,et al.  Pairwise Rotation Invariant Co-Occurrence Local Binary Pattern , 2014, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Hang Zhang,et al.  Friction from Reflectance: Deep Reflectance Codes for Predicting Physical Surface Properties from One-Shot In-Field Reflectance , 2016, ECCV.

[15]  Takayuki Okatani,et al.  Integrating deep features for material recognition , 2015, 2016 23rd International Conference on Pattern Recognition (ICPR).

[16]  Moshe Ben-Ezra,et al.  An LED-only BRDF measurement device , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Mehrtash Tafazzoli Harandi,et al.  Material Classification on Symmetric Positive Definite Manifolds , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[18]  Michael Weinmann,et al.  Material Classification Based on Training Data Synthesized Using a BTF Database , 2014, ECCV.

[19]  Edward H. Adelson,et al.  Recognizing Materials Using Perceptually Inspired Features , 2013, International Journal of Computer Vision.

[20]  Sang Chul Ahn,et al.  Surface reflectance estimation and segmentation from single depth image of ToF camera , 2016, Signal Process. Image Commun..

[21]  Wolfgang Heidrich,et al.  Material Classification Using Raw Time-of-Flight Measurements , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[23]  Mario Fritz,et al.  Recognizing Materials from Virtual Examples , 2012, ECCV.

[24]  Sabine Süsstrunk,et al.  Material-Based Object Segmentation Using Near-Infrared Information , 2010, Color Imaging Conference.

[25]  Subhransu Maji,et al.  Deep filter banks for texture recognition and segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Ramesh Raskar,et al.  Single view reflectance capture using multiplexed scattering and time-of-flight imaging , 2011, SA '11.

[27]  Kristin J. Dana,et al.  Deep TEN: Texture Encoding Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).