Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons

We study the recognition of surfaces made from different materials such as concrete, rug, marble, or leather on the basis of their textural appearance. Such natural textures arise from spatial variation of two surface attributes: (1) reflectance and (2) surface normal. In this paper, we provide a unified model to address both these aspects of natural texture. The main idea is to construct a vocabulary of prototype tiny surface patches with associated local geometric and photometric properties. We call these 3D textons. Examples might be ridges, grooves, spots or stripes or combinations thereof. Associated with each texton is an appearance vector, which characterizes the local irradiance distribution, represented as a set of linear Gaussian derivative filter outputs, under different lighting and viewing conditions.Given a large collection of images of different materials, a clustering approach is used to acquire a small (on the order of 100) 3D texton vocabulary. Given a few (1 to 4) images of any material, it can be characterized using these textons. We demonstrate the application of this representation for recognition of the material viewed under novel lighting and viewing conditions. We also illustrate how the 3D texton model can be used to predict the appearance of materials under novel conditions.

[1]  George S. Sebestyen,et al.  Pattern recognition by an adaptive process of sample set construction , 1962, IRE Trans. Inf. Theory.

[2]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[3]  G H Ball,et al.  A clustering technique for summarizing multivariate data. , 1967, Behavioral science.

[4]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[5]  J. Koenderink,et al.  Photometric Invariants Related to Solid Shape , 1980 .

[6]  B. Julesz.  Textons, the elements of texture perception, and their interactions , 1981, Nature.

[7]  Anil K. Jain,et al.  Markov Random Field Texture Models , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[9]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Rama Chellappa,et al.  Classification of textures using Gaussian Markov random fields , 1985, IEEE Trans. Acoust. Speech Signal Process..

[11]  L Sirovich,et al.  Low-dimensional Procedure for the Characterization of Human Faces , 1986 .

[12]  P Perona,et al.  Preattentive texture discrimination with early vision mechanisms , 1990 .

[13]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[14]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[15]  Anil K. Jain,et al.  Unsupervised texture segmentation using Gabor filters , 1990, 1990 IEEE International Conference on Systems, Man, and Cybernetics Conference Proceedings.

[16]  P. Vaidyanathan.  Multirate Systems And Filter Banks , 1992 .

[17]  Anil K. Jain,et al.  Texture classification and segmentation using multiresolution simultaneous autoregressive models , 1992, Pattern Recognit..

[18]  Jitendra Malik,et al.  A Computational Framework for Determining Stereo Correspondence from a Set of Linear Spatial Filters , 1991, ECCV.

[19]  Jitendra Malik,et al.  Computational framework for determining stereo correspondence from a set of linear spatial filters , 1992, Image Vis. Comput..

[20]  Mike J. Chantler.  Towards illuminant invariant texture classification , 1994 .

[21]  James R. Bergen,et al.  Texture Analysis: Representation and Matching , 1995, ICIAP.

[22]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[23]  Mike J. Chantler,et al.  Compensation of illuminant tilt variation for texture classification , 1995 .

[24]  Andrea J. van Doorn,et al.  Illuminance texture due to surface mesostructure , 1996 .

[25]  Joachim M. Buhmann,et al.  Non-parametric similarity measures for unsupervised texture segmentation and image retrieval , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[27]  Jitendra Malik,et al.  On perpendicular texture or: Why do we see more flowers in the distance? , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Paul A. Viola,et al.  Texture recognition using a non-parametric multi-scale statistical model , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[29]  David A. Forsyth,et al.  Shading primitives: finding folds and shallow grooves , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[30]  David J. Kriegman,et al.  Illumination cones for recognition under variable lighting: faces , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[31]  Shree K. Nayar,et al.  Histogram model for 3D textures , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[32]  J. Koenderink,et al.  Diffuse and specular reflectance from rough surfaces. , 1998, Applied optics.

[33]  Shree K. Nayar,et al.  3D Textured Surface Modeling , 1999 .

[34]  Shree K. Nayar,et al.  Reflectance and texture of real-world surfaces , 1999, TOGS.

[35]  Jitendra Malik,et al.  Recognizing surfaces using three-dimensional textons , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[36]  Carlo Tomasi,et al.  Texture-based image retrieval without segmentation , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[37]  Jitendra Malik,et al.  Textons, contours and regions: cue integration in image segmentation , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[38]  Shree K. Nayar,et al.  Correlation model for 3D texture , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[39]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[40]  William H. Press,et al.  Numerical recipes in C , 2002 .

[41]  Shree K. Nayar,et al.  Bidirectional Reflection Distribution Function of Thoroughly Pitted Surfaces , 1999, International Journal of Computer Vision.

[42]  D. Sagi,et al.  Gabor filters as texture discriminator , 1989, Biological Cybernetics.

[43]  Song-Chun Zhu,et al.  Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling , 1998, International Journal of Computer Vision.

[44]  Amnon Shashua,et al.  On Photometric Issues in 3D Visual Recognition from a Single 2D Image , 2004, International Journal of Computer Vision.

[45]  David J. Kriegman,et al.  What Is the Set of Images of an Object Under All Possible Illumination Conditions? , 1998, International Journal of Computer Vision.

[46]  Hiroshi Murase,et al.  Visual learning and recognition of 3-d objects from appearance , 2005, International Journal of Computer Vision.