Recovering shape and irradiance maps from rich dense texton fields

We describe a method that recovers an estimate of surface shape and of the irradiance field for a textured surface. The method assumes the surface is viewed in scaled orthography, and we demonstrate the appropriateness of this assumption. Our method uses interest points to obtain the locations of putative texton instances, clusters the textons into types, and then uses an autocalibration method to recover the frontal appearance of each texton model. This yields (a) a dense set of normal estimates, each up to a two-fold ambiguity, (b) a dense set of irradiance estimates and (c) whether each instance is, in fact, an instance of the relevant texton. Because we are able to obtain a very large number of instances of a large number of different textons, this information is obtained at sites very closely spaced in the image. As a result, we need only a simple smoothness constraint to reconstruct a surface model, using EM to resolve the normal ambiguity. We show results on images of real scenes, comparing our reconstructions with those obtained using other methods and demonstrating the accuracy of both the recovered shape and the irradiance estimate.

[1]  John C. Platt,et al.  Elastically deformable models , 1987, SIGGRAPH.

[2]  Andrew Blake,et al.  Shape from Texture: Estimation, Isotropy and Moments , 1990, Artif. Intell..

[3]  Jonas Gårding,et al.  Shape from Texture for Smooth Curved Surfaces , 1992, ECCV.

[4]  Jonas Gårding,et al.  Surface orientation and curvature from differential texture distortion , 1995, Proceedings of IEEE International Conference on Computer Vision.

[5]  Jitendra Malik,et al.  Detecting, localizing and grouping repeated scene elements from an image , 1996, ECCV.

[6]  Cordelia Schmid,et al.  Local Grayvalue Invariants for Image Retrieval , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Jitendra Malik,et al.  Surface orientation from texture: Isotropy or homogeneity (or both)? , 1997, Vision Research.

[8]  Bill Triggs,et al.  Autocalibration from Planar Scenes , 1998, ECCV.

[9]  Jitendra Malik,et al.  Textons, contours and regions: cue integration in image segmentation , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[10]  Stéphane Mallat,et al.  Shape from texture through deformations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[12]  Stéphane Mallat,et al.  The Texture Gradient Equation for Recovering Shape from Texture , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  A. U.S.,et al.  Recovering Surface Shape and Orientation from Texture , 2002 .

[14]  Sparse Texture Representation Using Affine-Invariant Neighborhoods CVPR Paper , 2003 .

[15]  Cordelia Schmid,et al.  Affine-invariant local descriptors and neighborhood statistics for texture recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[16]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[17]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[18]  Jitendra Malik,et al.  Computing Local Surface Orientation and Shape from Texture for Curved Surfaces , 1997, International Journal of Computer Vision.

[19]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Ronald Fedkiw,et al.  Robust treatment of collisions, contact and friction for cloth animation , 2002, SIGGRAPH Courses.

[21]  David A. Forsyth,et al.  Shape from Texture without Boundaries , 2002, International Journal of Computer Vision.