Building 3-D Human Face Models from Two Photographs

This paper presents a working system for building 3-D human face models from two photographs. Rather than using expensive 3-D scanners, we show that frontal face models can be faithfully reconstructed from two photographs taken by consumer digital cameras in a totally non-invasive setup. We first rectify the image pair so that corresponding epipolar lines become coincident, by computing a dual point transformation. We then address the correspondence problem by converting it into a maximal surface extraction problem, which is then solved efficiently. The method effectively removes local extrema. Finally, a Euclidean reconstruction is achieved with the help of a novel factorization method for perspective cameras. Most of the computational steps are conducted in projective space. Euclidean information is introduced only at the last stage. This sets apart our system from the traditional ones which begin with metric information by using carefully calibrated cameras. We have collected a bank of face pairs to test our system, and are satisfied with its performance. Results from this image database are demonstrated.

[1]  Wilfried Brauer,et al.  Intensity- and Gradient-Based Stereo Matching Using Hierarchical Gaussian Basis Functions , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Koji Koyamada,et al.  Automatic Isosurface Propagation Using an Extrema Graph and Sorted Boundary Cell Lists , 1995, IEEE Trans. Vis. Comput. Graph..

[3]  Qian Chen,et al.  Efficient iterative solution to M-view projective reconstruction problem , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[4]  Gérard G. Medioni,et al.  Inference of Integrated Surface, Curve, and Junction Descriptions From Sparse 3D Data , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Gérard G. Medioni,et al.  A semi-automatic system to infer complex 3-D shapes from photographs , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[6]  Ingemar J. Cox,et al.  A Maximum Likelihood Stereo Algorithm , 1996, Comput. Vis. Image Underst..

[7]  Thierry Viéville,et al.  Canonic Representations for the Geometries of Multiple Projective Views , 1994, ECCV.

[8]  Henrique S. Malvar,et al.  Making Faces , 2019, Topoi.

[9]  Martin A. Fischler,et al.  Computational Stereo , 1982, CSUR.

[10]  B. Julesz,et al.  A disparity gradient limit for binocular fusion. , 1980, Science.

[11]  Takeo Kanade,et al.  Stereo by Intra- and Inter-Scanline Search Using Dynamic Programming , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Qian Chen,et al.  A volumetric stereo matching method: application to image-based modeling , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[13]  Jake K. Aggarwal,et al.  Structure from stereo-a review , 1989, IEEE Trans. Syst. Man Cybern..

[14]  Gérard G. Medioni,et al.  Integrated surface, curve and junction inference from sparse 3-D data sets , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[15]  Narendra Ahuja,et al.  Surfaces from Stereo: Integrating Feature Matching, Disparity Estimation, and Contour Detection , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[17]  Mi-Suen Lee,et al.  Inferring segmented surface description from stereo data , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[18]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[19]  Davi Geiger,et al.  Occlusions, Discontinuities, and Epipolar Lines in Stereo , 1998, ECCV.

[20]  Greg Welch,et al.  The office of the future: a unified approach to image-based modeling and spatially immersive displays , 1998, SIGGRAPH.

[21]  Takeo Kanade,et al.  A Paraperspective Factorization Method for Shape and Motion Recovery , 1994, ECCV.

[22]  O. Faugeras.  Stratification of three-dimensional vision: projective, affine, and metric representations , 1995 .

[23]  B. Julesz,et al.  Modifications of the Classical Notion of Panum's Fusional Area , 1980, Perception.

[24]  Andrew Zisserman,et al.  Geometric invariance in computer vision , 1992 .

[25]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[26]  Ingemar J. Cox,et al.  A maximum-flow formulation of the N-camera stereo correspondence problem , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).