Study of Volumetric Methods for Face Reconstruction

This paper presents an early study about a reconstruction method that extends previous space carving methods to handle all characteristics of human faces: complex non-Lambertian materials, untextured areas, highly detailed geometry, etc. This face study is seen as a preliminary step to a more general framework for non-Lambertian reconstruction. We therefore avoid specific techniques like parameterized face models. Since we expose our early studies, we mainly present the related existing work and discuss the pros and cons of each approach. We aim at discerning the strength and weaknesses of the classical tools in order to adapt and improve them to handle non-Lambertian materials while overcoming their limitations. We show our first results which are promising and validate our global approach. Throughout the discussion, we raise several questions that identify complex issues related to our goal. We provide some hints that pave the way for a better understanding of the global problem. We are confident in that further studies will lead to significant improvements over existing methods.

[1]  Paulo R. S. Mendonça,et al.  Camera Pose Estimation and Reconstruction from Image Profiles under Circular Motion , 2000, ECCV.

[2]  K. Torrance,et al.  Image-Based BRDF Measurement , 1999 .

[3]  David J. Kriegman,et al.  Illumination-based image synthesis: creating novel images of human faces under differing pose and lighting , 1999, Proceedings IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes (MVIEW'99).

[4]  Javier Cuenca,et al.  Heuristics for work distribution of a homogeneous parallel dynamic programming scheme on heterogeneous systems , 2004, Third International Symposium on Parallel and Distributed Computing/Third International Workshop on Algorithms, Models and Tools for Parallel Computing on Heterogeneous Networks.

[5]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[6]  Olivier D. Faugeras,et al.  Variational principles, surface evolution, PDEs, level set methods, and the stereo problem , 1998, IEEE Trans. Image Process..

[7]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Zicheng Liu,et al.  Model-based bundle adjustment with application to face modeling , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[9]  Shree K. Nayar,et al.  A Theory of Specular Surface Geometry , 2004, International Journal of Computer Vision.

[10]  Stephen Lin,et al.  Multibaseline stereo in the presence of specular reflections , 2002, Object recognition supported by user interaction for service robots.

[11]  Rafael Mayo,et al.  Parallelization of the GNU Scientific Library on heterogeneous systems , 2004, Third International Symposium on Parallel and Distributed Computing/Third International Workshop on Algorithms, Models and Tools for Parallel Computing on Heterogeneous Networks.

[12]  Richard Szeliski,et al.  Stereo Matching with Transparency and Matting , 1999, International Journal of Computer Vision.

[13]  Steven M. Seitz,et al.  Photorealistic Scene Reconstruction by Voxel Coloring , 1997, International Journal of Computer Vision.

[14]  Vladimir Kolmogorov,et al.  Multi-camera Scene Reconstruction via Graph Cuts , 2002, ECCV.

[15]  Wojciech Matusik,et al.  Polyhedral Visual Hulls for Real-Time Rendering , 2001, Rendering Techniques.

[16]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[17]  Long Quan,et al.  A Volumetric Reconstruction Method from Multiple Calibrated Views using Global Graph Cut Optimization , 2003 .

[18]  O. Faugeras,et al.  Variational principles, surface evolution, PDE's, level set methods and the stereo problem , 1998, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[19]  Olivier D. Faugeras,et al.  Using Extremal Boundaries for 3-D Object Modeling , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Luis Pastor,et al.  Dynamic load balancing in heterogeneous clusters , 2004, Parallel and Distributed Computing and Networks.

[21]  Richard Szeliski,et al.  Robust Shape Recovery from Occluding Contours Using a Linear Smoother , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Henrique S. Malvar,et al.  Making Faces , 2019, Topoi.

[23]  Thomas Malzbender,et al.  Improved Voxel Coloring Via Volumetric Optimization , 2001 .

[24]  Shree K. Nayar,et al.  Stereo in the presence of specular reflection , 1995, Proceedings of IEEE International Conference on Computer Vision.

[25]  Thomas Malzbender,et al.  A Survey of Methods for Volumetric Scene Reconstruction from Photographs , 2001, VG.

[26]  Stephen Lin,et al.  Separation of diffuse and specular reflection in color images , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[27]  Thomas Malzbender,et al.  Generalized Voxel Coloring , 1999, Workshop on Vision Algorithms.

[28]  Stephen Lin,et al.  Diffuse-Specular Separation and Depth Recovery from Image Sequences , 2002, ECCV.

[29]  H. Ishikawa.  Global Optimization Using Embedded Graphs , 2000 .

[30]  Long Quan,et al.  Quasi-Dense Reconstruction from Image Sequence , 2002, ECCV.

[31]  Kiriakos N. Kutulakos.  Approximate N-View Stereo , 2000, ECCV.

[32]  David J. Kriegman,et al.  Beyond Lambert: reconstructing surfaces with arbitrary BRDFs , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.