A Statistical Approach to Material Classification Using Image Patch Exemplars

In this paper, we investigate material classification from single images obtained under unknown viewpoint and illumination. It is demonstrated that materials can be classified using the joint distribution of intensity values over extremely compact neighborhoods (starting from as small as 3times3 pixels square) and that this can outperform classification using filter banks with large support. It is also shown that the performance of filter banks is inferior to that of image patches with equivalent neighborhoods. We develop novel texton-based representations which are suited to modeling this joint neighborhood distribution for Markov random fields. The representations are learned from training images and then used to classify novel images (with unknown viewpoint and lighting) into texture classes. Three such representations are proposed and their performance is assessed and compared to that of filter banks. The power of the method is demonstrated by classifying 2,806 images of all 61 materials present in the Columbia-Utrecht database. The classification performance surpasses that of recent state-of-the-art filter bank-based classifiers such as Leung and Malik (IJCV 01), Cula and Dana (IJCV 04), and Varma and Zisserman (IJCV 05). We also benchmark performance by classifying all of the textures present in the UIUC, Microsoft Textile, and San Francisco outdoor data sets. We conclude with discussions on why features based on compact neighborhoods can correctly discriminate between textures with large global structure and why the performance of filter banks is not superior to that of the source image patches from which they were derived.

[1]  David G. Stork,et al.  Pattern Classification , 1973 .

[2]  B Julesz,et al.  Inability of Humans to Discriminate between Visual Textures That Agree in Second-Order Statistics—Revisited , 1973, Perception.

[3]  B. Julesz.  Textons, the elements of texture perception, and their interactions , 1981, Nature.

[4]  David Donovan Garber,et al.  Computational models for texture analysis and texture synthesis , 1981 .

[5]  Anil K. Jain,et al.  Markov Random Field Texture Models , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  M. Unser.  Local linear transforms for texture measurements , 1986 .

[8]  E. Adelson,et al.  Early vision and texture perception , 1988, Nature.

[9]  P Perona,et al.  Preattentive texture discrimination with early vision mechanisms. , 1990, Journal of the Optical Society of America. A, Optics and image science.

[10]  graphics , 1991 .

[11]  Kris Popat,et al.  Novel cluster-based probability model for texture synthesis, classification, and compression , 1993, Other Conferences.

[12]  Shih-Fu Chang,et al.  Transform features for texture classification and discrimination in large image databases , 1994, Proceedings of 1st International Conference on Image Processing.

[13]  Pietro Perona,et al.  Rotation invariant texture recognition using a steerable pyramid , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[14]  B. S. Manjunath,et al.  Rotation-invariant texture classification using modified Gabor filters , 1995, Proceedings., International Conference on Image Processing.

[15]  Yann LeCun,et al.  Transformation Invariance in Pattern Recognition-Tangent Distance and Tangent Propagation , 1996, Neural Networks: Tricks of the Trade.

[16]  Anil K. Jain,et al.  Learning Texture Discrimination Masks , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[18]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[19]  Shree K. Nayar,et al.  Reflectance and texture of real-world surfaces , 1999, TOGS.

[20]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[21]  Trygve Randen,et al.  Filtering for Texture Classification: A Comparative Study , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Bernard Victorri,et al.  Transformation invariance in pattern recognition: Tangent distance and propagation , 2000 .

[23]  Alan L. Yuille,et al.  Statistical cues for domain specific image segmentation with performance analysis , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[24]  Luc Van Gool,et al.  A Compact Model for Viewpoint Dependent Texture Synthesis , 2000, SMILE.

[25]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[26]  Stan Z. Li,et al.  Markov Random Field Modeling in Image Analysis , 2001, Computer Science Workbench.

[27]  Cordelia Schmid,et al.  Constructing models for content-based image retrieval , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[28]  Jitendra Malik,et al.  Learning affinity functions for image segmentation: combining patch-based and gradient-based approaches , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[29]  Andrew Zisserman,et al.  Texture classification: are filter banks necessary? , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[30]  Barbara Caputo,et al.  Recognition with local features: the kernel recipe , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[31]  Ilan Shimshoni,et al.  Mean shift based clustering in high dimensions: a texture classification example , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[32]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[33]  Jinbo Bi,et al.  Dimensionality Reduction via Sparse Support Vector Machines , 2003, J. Mach. Learn. Res..

[34]  Yanxi Liu,et al.  The Promise and Perils of Near-Regular Texture , 2004, International Journal of Computer Vision.

[35]  M. Varma.  Statistical Approaches to Texture Classification , 2004 .

[36]  Jitendra Malik,et al.  Contour and Texture Analysis for Image Segmentation , 2001, International Journal of Computer Vision.

[37]  Kristin J. Dana,et al.  3D Texture Recognition Using Bidirectional Feature Histograms , 2004, International Journal of Computer Vision.

[38]  Cordelia Schmid,et al.  Weakly Supervised Learning of Visual Models and Its Application to Content-Based Retrieval , 2004, International Journal of Computer Vision.

[39]  Yanxi Liu,et al.  Near-regular texture analysis and manipulation , 2004, SIGGRAPH 2004.

[40]  Andrew Zisserman,et al.  Unifying statistical texture classification frameworks , 2004, Image Vis. Comput..

[41]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[42]  Andrew Zisserman,et al.  A Statistical Approach to Texture Classification from Single Images , 2004, International Journal of Computer Vision.

[43]  Jitendra Malik,et al.  Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons , 2001, International Journal of Computer Vision.

[44]  Mario Fritz,et al.  On the Significance of Real-World Conditions for Material Classification , 2004, ECCV.

[45]  Tony Lindeberg,et al.  Feature Detection with Automatic Scale Selection , 1998, International Journal of Computer Vision.

[46]  Paul A. Viola,et al.  Boosting Image Retrieval , 2004, International Journal of Computer Vision.

[47]  Barbara Caputo,et al.  Class-Specific Material Categorisation , 2005, ICCV.

[48]  Christopher J. C. Burges,et al.  Geometric Methods for Feature Extraction and Dimensional Reduction , 2005 .

[49]  Cordelia Schmid,et al.  A sparse texture representation using local affine regions , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[51]  Robert E. Broadhurst.  Statistical Estimation of Histogram Variation for Texture Classification , 2005 .

[52]  Lior Wolf,et al.  Patch-Based Texture Edges and Segmentation , 2006, ECCV.

[53]  Yong Xu,et al.  A Projective Invariant for Textures , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[54]  Alexei A. Efros,et al.  Discovering Texture Regularity as a Higher-Order Correspondence Problem , 2006, ECCV.

[55]  Yanxi Liu,et al.  Quantitative Evaluation of Near Regular Texture Synthesis Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[56]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[57]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[58]  Manik Varma,et al.  Locally Invariant Fractal Features for Statistical Texture Classification , 2007, 2007 IEEE 11th International Conference on Computer Vision.