Statistical Approach to Shape from Shading : Reconstruction of 3 D Face Surfaces from Single 2 D

The human visual system is proocient in perceiving three-dimensional shape from the shading patterns in a two-dimensional image. How it does this is not well understood and continues to be a question of fundamental and practical interest. In this paper we present a new quantitative approach to shape-from-shading that may provide some answers. We suggest that the brain, through evolution or prior experience, has discovered that objects can be classiied into lower-dimensional object-classes as to their shape. Extraction of shape from shading is then equivalent to the much simpler problem of parameter estimation in a low dimensional space. We carry out this proposal for an important class of 3D objects; human heads. From an ensemble of several hundred laser-scanned 3D heads, we use principal component analysis to derive a low-dimensional parameterization of head shape space. An algorithm for solving shape-from-shading using this representation is presented. It works well even on real images where it is able to recover the 3D surface for a given person, maintaining facial detail and identity, from a single 2D image of their face. This algorithm has applications in face recognition and animation.

[1]  Berthold K. P. Horn.  SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW , 1970 .

[2]  R. Gregory.  The intelligent eye , 1970 .

[3]  E. Simmons.  Diffuse reflectance spectroscopy: a comparison of the theories. , 1975, Applied optics.

[4]  Robert B. Lawson,et al.  Human stereopsis : a psychophysical analysis , 1976 .

[5]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[6]  Katsushi Ikeuchi,et al.  Numerical Shape from Shading and Occluding Boundaries , 1981, Artif. Intell..

[7]  Robert J. Woodham,et al.  Analysing Images of Curved Surfaces , 1981, Artif. Intell..

[8]  A. Pentland.  Finding the illuminant direction , 1982 .

[9]  E. Mingolla,et al.  Perception of surface curvature and direction of illumination from patterns of shading. , 1983, Journal of experimental psychology. Human perception and performance.

[10]  Alex Pentland,et al.  Local Shading Analysis , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Azriel Rosenfeld,et al.  Improved Methods of Estimating Shape from Shading Using the Light Source Coordinate System , 1985, Artif. Intell..

[12]  Michael J. Brooks,et al.  Shape and Source from Shading , 1985, IJCAI.

[13]  L. Sirovich.  Turbulence and the dynamics of coherent structures. II. Symmetries and transformations , 1987 .

[14]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[15]  Terrence J. Sejnowski,et al.  Network model of shape-from-shading: neural function arises from both receptive and projective fields , 1988, Nature.

[16]  V. S. Ramachandran,et al.  Perception of shape from shading , 1988, Nature.

[17]  Vilayanur S. Ramachandran,et al.  Theories of Perception. , 1951 .

[18]  George Wolberg,et al.  Digital image warping , 1990 .

[19]  Lawrence Sirovich,et al.  Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Rama Chellappa,et al.  Estimation of illuminant direction, albedo, and shape from shading , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  J. Oliensis,et al.  Shape from shading as a partially well-constrained problem , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Lawrence Sirovich,et al.  Management and Analysis of Large Scientific Datasets , 1992 .

[23]  The International Journal of Supercomputer Applications— , 1992 .

[24]  Thomas Ertl,et al.  Computer Graphics - Principles and Practice, 3rd Edition , 2014 .