Analysis of shape from shading algorithms for fast and realistic 3D face reconstruction APCCAS2002

The aim of this paper is to attain a fast and realistic 3D face reconstruction, when only a limited set of images taken from multiple views, is available. Instead of relying merely on 2D images as suggested by image-based modeling approaches, we investigate several shape-from-shading (SFS) techniques, which have been studied extensively in computer vision research. This is because we believe that a nearly complete set of images required by image-based modeling techniques is rarely available in real world applications. In this paper, we investigate the effectiveness of three different SFS algorithms to provide partial 3D shapes of the face to be reconstructed. Each algorithm is selected from three different classes of SFS technique, i.e., linear, propagation, and minimization approaches. The reconstruction process is performed by our novel neural network learning scheme, which is able to successively refine the polygon vertices parameter of an initial 3D shape, based on depth maps of several calibrated images. To evaluate the reconstruction result based on those SFS techniques, we measure average vertex-error and pixel-error compared to actual 3D data obtained by 3D scanner device. We also compared these result with those obtained by using only 2D images. In addition, we also measure total computational time needed in the reconstruction process.

[1]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[2]  Mubarak Shah,et al.  Iterative shape recovery from multiple images , 1997, Image and Vision Computing.

[3]  Zhengyou Zhang,et al.  Image-Based Geometrically-Correct Photorealistic Scene/Object Modeling (IBPhM): A Review , 1998, ACCV.

[4]  Jed Lengyel,et al.  The Convergence of Graphics and Vision , 1998, Computer.

[5]  Peter Eisert,et al.  Multi-view image coding with depth maps and 3-D geometry for prediction , 2001 .

[6]  Andrea Fusiello.  Image-based Rendering * , 2003 .

[7]  Rama Chellappa,et al.  Estimation of illuminant direction, albedo, and shape from shading , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Takaaki Akimoto,et al.  Automatic creation of 3D facial models , 1993, IEEE Computer Graphics and Applications.

[9]  Mubarak Shah,et al.  Shape from shading using linear approximation , 1994, Image Vis. Comput..

[10]  Paul A. Griffin,et al.  Statistical Approach to Shape from Shading: Reconstruction of Three-Dimensional Face Surfaces from Single Two-Dimensional Images , 1996, Neural Computation.

[11]  Pascal Fua,et al.  3D stereo reconstruction of human faces driven by differential constraints , 2000, Image Vis. Comput..

[12]  Itsuo Kumazawa,et al.  3D Shape Reconstruction from Multiple Silhouettes: Generalization from Few Views by Neural Network Learning , 2001, IWVF.

[13]  Itsuo Kumazawa,et al.  Face Reconstruction from Shading Using Smooth Projected Polygon Representation NN , 2002 .

[14]  Alex Pentland,et al.  A simple algorithm for shape from shading , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Itsuo Kumazawa,et al.  A scheme for reconstructing face from shading using smooth projected polygon representation NN , 2002, Proceedings. International Conference on Image Processing.

[16]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Sing Bing Kang,et al.  Survey of image-based rendering techniques , 1998, Electronic Imaging.