3D face reconstruction using images from cameras with varying parameters

In this paper, we present a new technique of 3D face reconstruction from a sequence of images taken with cameras having varying parameters without the need to grid. This method is based on the estimation of the projection matrices of the cameras from a symmetry property which characterizes the face, these projections matrices are used with points matching in each pair of images to determine the 3D points cloud, subsequently, 3D mesh of the face is constructed with 3D Crust algorithm. Lastly, the 2D image is projected on the 3D model to generate the texture mapping. The strong point of the proposed approach is to minimize the constraints of the calibration system: we calibrated the cameras from a symmetry property which characterizes the face, this property gives us the opportunity to know some points of 3D face in a specific well-chosen global reference, to formulate a system of linear and nonlinear equations according to these 3D points, their projection in the image plan and the elements of the projections matrix. Then to solve these equations, we use a genetic algorithm which consists of finding the global optimum without the need of the initial estimation and allows to avoid the local minima of the formulated cost function. Our study is conducted on real data to demonstrate the validity and the performance of the proposed approach in terms of robustness, simplicity, stability and convergence.

[1]  Paul A. Griffin,et al.  Statistical Approach to Shape from Shading: Reconstruction of Three-Dimensional Face Surfaces from Single Two-Dimensional Images , 1996, Neural Computation.

[2]  Ansari A-Nasser,et al.  3D Face Mesh Modeling for 3D Face Recognition , 2009 .

[3]  Sunghee Choi,et al.  The power crust , 2001, SMA '01.

[4]  Shinn-Ying Ho,et al.  Facial modeling from an uncalibrated face image using a coarse-to-fine genetic algorithm , 2001, Pattern Recognit..

[5]  Volkan Atalay,et al.  Delaunay Triangulation based 3D Human Face Modeling from Uncalibrated Images , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[6]  Sunghee Choi,et al.  The power crust, unions of balls, and the medial axis transform , 2001, Comput. Geom..

[7]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[8]  Ioannis Pitas,et al.  3D Human Face Modelling From Uncalibrated Images Using Spline Based Deformation , 2008, VISAPP.

[9]  Bin Li,et al.  Identification of Eye Movements from Non-frontal Face Images for Eye-controlled Systems , 2014, Int. J. Autom. Comput..

[10]  Cordelia Schmid,et al.  Comparing and evaluating interest points , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[11]  Zhengxu Zhao,et al.  The skin deformation of a 3D virtual human , 2009, Int. J. Autom. Comput..

[12]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[13]  A-Nasser Ansari,et al.  3D face modeling using two orthogonal views and a generic face model , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[14]  Nina Amenta.  The crust algorithm for 3D surface reconstruction , 1999, SCG '99.

[15]  Mongi A. Abidi,et al.  Automated Scene-Specific Selection of Feature Detectors for 3D Face Reconstruction , 2007, ISVC.

[16]  Khalid Satori,et al.  Camera Self Calibration with Varying Parameters by an Unknown Three Dimensional Scene Using the Improved Genetic Algorithm , 2015 .

[17]  Hideo Saito,et al.  PCA-based 3D shape reconstruction of human foot using multiple viewpoint cameras , 2008, Int. J. Autom. Comput..

[18]  Chun Chen,et al.  New algorithm for 3D facial model reconstruction and its application in virtual reality , 2004, Journal of Computer Science and Technology.

[19]  Kang Ryoung Park,et al.  A SfM-based 3D face reconstruction method robust to self-occlusion by using a shape conversion matrix , 2011, Pattern Recognit..

[20]  Li Wang,et al.  Feature Representation for Facial Expression Recognition Based on FACS and LBP , 2014, Int. J. Autom. Comput..

[21]  Varin Chouvatut,et al.  Face reconstruction and camera pose using Multi-dimensional descent , 2011 .

[22]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[23]  鄭文昌,et al.  3D Face Reconstruction Using the Stereo Camera and Neural Network Regression , 2009 .

[24]  Pascal Fua,et al.  3D face modeling from stereo and differential constraints , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[25]  Panos Nasiopoulos,et al.  The Effect of Frame Rate on 3D Video Quality and Bitrate , 2015, 1803.04826.

[26]  Horst Bunke,et al.  Classifying range images of human faces with Hausdorff distance , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.