Combining Cues: Shape from Shading and Texture

We demonstrate a method for reconstructing the shape of a deformed surface from a single view. After decomposing an image into irradiance and albedo components, we combine normal cues from shading and texture to produce a field of unambiguous normals. Using these normals, we reconstruct the 3D geometry. Our method works in two regimes: either requiring the frontal appearance of the texture or building it automatically from a series of images of the deforming texture. We can recover geometry with errors below four percent of object size on arbitrary textures, and estimate specific geometric parameters using a custom texture even more accurately.

[1]  Thomas S. Huang,et al.  Motion and structure from orthographic projections , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[2]  Rangasami L. Kashyap,et al.  3-D Shape from a Shaded and Textural Surface Image , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Berthold K. P. Horn SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW , 1970 .

[4]  Xavier Provot,et al.  Deformation Constraints in a Mass-Spring Model to Describe Rigid Cloth Behavior , 1995 .

[5]  S. Ullman The Interpretation of Visual Motion , 1979 .

[6]  David A. Forsyth,et al.  Reflections on Shading , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  D. Forsyth,et al.  Deforming Objects Provide Better Camera Calibration , 2005 .

[8]  Angeline M. Loh,et al.  Shape from Non-homogeneous, Non-stationary, Anisotropic, Perspective Texture , 2005, BMVC.

[9]  Vincent Lepetit,et al.  Real-time nonrigid surface detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  David A. Forsyth,et al.  Recovering shape and irradiance maps from rich dense texton fields , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[11]  David A. Forsyth,et al.  Shape from Texture without Boundaries , 2002, International Journal of Computer Vision.

[12]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Dmitry B. Goldgof,et al.  Nonrigid motion analysis based on dynamic refinement of finite element models , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[14]  David A. Forsyth,et al.  Retexturing Single Views Using Texture and Shading , 2006, ECCV.

[15]  David A. Forsyth,et al.  Shape from texture and integrability , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[16]  Min Chen,et al.  Recovering Local Shape of a Mirror Surface from Reflection of a Regular Grid , 2004, ECCV.