Shape from Texture without Boundaries

We describe a shape from texture method that constructs an estimate of surface geometry using only the deformation of individual texture elements. Our method does not need to use either the boundary of the observed surface or any assumption about the overall distribution of elements.The method assumes that surface texture elements are drawn from a number of different types, each of fixed shape. Neither the shape of the elements nor the number of types need be known in advance. We show that, with this assumption and assuming a generic, scaled orthographic view and texture, each type of texture element can be reconstructed in a frontal coordinate system from image instances. Interest-point methods supply a method of simultaneously obtaining instances of each texture element automatically and defining each type of element. Furthermore, image instances that have been marked in error can be identified and ignored using the Expectation-Maximization algorithm. A further EM procedure yields a surface reconstruction and a relative irradiance map from the data. We provide numerous examples of reconstructions for images of real scenes, show a comparison between our reconstruction and range maps, and demonstrate that the reconstructions display geometric and irradiance phenomena that can be observed in the original image.

[1]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[2]  James V. Stone,et al.  Adaptive Scale Filtering: A General Method for Obtaining Shape From Texture , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Bruce J. Schachter,et al.  Model-Based Texture Measures , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Cordelia Schmid,et al.  Local Grayvalue Invariants for Image Retrieval , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Bill Triggs,et al.  Autocalibration from Planar Scenes , 1998, ECCV.

[6]  Narendra Ahuja,et al.  Surveyor's Forum: Image Models , 1983, CSUR.

[7]  Alexander C. Berg,et al.  Who's In the Picture , 2004, NIPS 2004.

[8]  Lorenzo Torresani,et al.  Tracking and modeling non-rigid objects with rank constraints , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[9]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[10]  David E. Breen,et al.  Cloth modeling and animation , 2000 .

[11]  Ronald Fedkiw,et al.  Robust treatment of collisions, contact and friction for cloth animation , 2002, SIGGRAPH Courses.

[12]  Leif H. Finkel,et al.  A shape-from-texture algorithm based on human visual psychophysics , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Andrew W. Fitzgibbon,et al.  On Affine Invariant Clustering and Automatic Cast Listing in Movies , 2002, ECCV.

[14]  Jonas Gårding,et al.  Surface orientation and curvature from differential texture distortion , 1995, Proceedings of IEEE International Conference on Computer Vision.

[15]  David A. Forsyth,et al.  Shape from texture and integrability , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[16]  John Krumm,et al.  Shape from periodic texture using the spectrogram , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Paul A. Viola,et al.  Learning from one example through shared densities on transforms , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[18]  David A. Forsyth.  Shape from Texture without Boundaries , 2002, ECCV.

[19]  Wolfgang Heidrich,et al.  Cloth Motion Capture , 2003, Comput. Graph. Forum.

[20]  Daryl J. Daley,et al.  An Introduction to the Theory of Point Processes , 2013 .

[21]  Stéphane Mallat,et al.  Shape from texture through deformations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[22]  Jitendra Malik,et al.  Textons, contours and regions: cue integration in image segmentation , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[23]  Stéphane Mallat,et al.  The Texture Gradient Equation for Recovering Shape from Texture , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Narendra Ahuja,et al.  Random pattern generation processes , 1979 .

[26]  C.-C. Jay Kuo,et al.  Direct shape from texture using a parametric surface model and an adaptive filtering technique , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[27]  Joseph L. Mundy,et al.  Repeated Structures: Image Correspondence Constraints and 3D Structure Recovery , 1993, Applications of Invariance in Computer Vision.

[28]  Jitendra Malik,et al.  Surface orientation from texture: Isotropy or homogeneity (or both)? , 1997, Vision Research.

[29]  Andrew Blake,et al.  Shape from Texture: Estimation, Isotropy and Moments , 1990, Artif. Intell..

[30]  Alan C. Bovik,et al.  Shape from Texture Using Local Spectral Moments , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  David Pritchard,et al.  Cloth Parameters and Motion Capture , 2003 .

[32]  Govind Sharma.  Surveyor's Forum: Image Models , 1983 .

[34]  Jan J. Koenderink,et al.  Bidirectional Texture Contrast Function , 2002, ECCV.

[35]  J. Koenderink,et al.  Geometrical modes as a general method to treat diffuse interreflections in radiometry , 1983 .

[36]  Cordelia Schmid,et al.  Affine-invariant local descriptors and neighborhood statistics for texture recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[37]  John Krumm,et al.  Local Spatial Frequency Analysis for Computer Vision , 1990 .

[38]  John C. Platt,et al.  Elastically deformable models , 1987, SIGGRAPH.

[39]  J. Koenderink,et al.  Optical properties (bidirectional reflection distribution functions) of velvet. , 1998, Applied optics.

[40]  Jitendra Malik,et al.  Computing Local Surface Orientation and Shape from Texture for Curved Surfaces , 1997, International Journal of Computer Vision.

[41]  Norman I. Badler,et al.  SIGGRAPH VRML 3D Ph.D Conetree , 1998, COMG.

[42]  Andrew P. Witkin,et al.  Large steps in cloth simulation , 1998, SIGGRAPH.

[43]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[44]  Ruzena Bajcsy,et al.  Texture gradient as a depth cue , 1976 .

[45]  Narendra Ahuja,et al.  Image Models , 1981, CSUR.

[46]  John Krumm,et al.  Local spatial frequency analysis of image texture , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[47]  Andrew Zisserman,et al.  Geometric Grouping of Repeated Elements within Images , 1999, Shape, Contour and Grouping in Computer Vision.

[48]  David A. Forsyth,et al.  Whos In the Picture , 2004, NIPS.

[49]  Jitendra Malik,et al.  Detecting, localizing and grouping repeated scene elements from an image , 1996, ECCV.

[50]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[51]  Yee Whye Teh,et al.  Faces and names in the news , 2004, CVPR 2004.

[52]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[53]  Jonas Gårding,et al.  Shape from Texture for Smooth Curved Surfaces , 1992, ECCV.

[54]  A. U.S.,et al.  Recovering Surface Shape and Orientation from Texture , 2002 .

[55]  Rangasami L. Kashyap,et al.  3-D Shape from a Shaded and Textural Surface Image , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[56]  Ronald Fedkiw,et al.  Simulation of clothing with folds and wrinkles , 2003, SCA '03.