A Novel Approach of 3D Reconstruction of Human Face Using Monocular Camera

Three-dimensional model acquisition of an object is essential in many multimedia applications. Constructing three-dimensional models of objects from two-dimensional images is an old problem in the area of computer vision. There are many publications and our approach is specifically designed for constructing the depth map of a human face on frontal view, based on the head movement in a monocular setting. Our approach does not require a predefined face model. In this paper, along with the front view image of the user, three additional images with various head movement are captured, and the head pose is then calculated. The depth map is calculated through a triangular mesh, which the nodes on the mesh are the feature points that we calculate the depth with. Through image registration process, the feature points on the front view image are mapped to the other three images. Based on the head pose and the newly mapped coordinate, we could calculate the depth of the feature point. The depth results calculated from each of the three images are combined together to find the final depth value. Our approach does not require additional hardware, or predefined models. The result is not as good as we wished, but suggests avenues for future.

[1]  Wojciech Matusik,et al.  3D Reconstruction Using Labeled Image Regions , 2003, Symposium on Geometry Processing.

[2]  Jezekiel Ben-Arie,et al.  Recovery of 3-D face structure using recognition , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[3]  Ulrich Neumann,et al.  SYNTHESIS OF 3D FACES , 1999 .

[4]  Shinn-Ying Ho,et al.  An analytic solution for the pose determination of human faces from a monocular image , 1998, Pattern Recognit. Lett..

[5]  Kentaro Toyama,et al.  Manipulation of Video Eye Gaze and Head Orientation for Video Teleconferencing , 1999 .

[6]  Sotiris Malassiotis,et al.  Coding of video-conference stereo image sequences using 3D models , 1997, Signal Process. Image Commun..

[7]  Roberto Cipolla,et al.  Determining the gaze of faces in images , 1994, Image Vis. Comput..

[8]  Hervé Delingette,et al.  Low cost 3D face acquisition and modeling , 2001, Proceedings International Conference on Information Technology: Coding and Computing.

[9]  Larry S. Davis,et al.  Foundations of Image Understanding , 2001 .

[10]  Jean-Philippe Tarel,et al.  From 2D images to 3D face geometry , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[11]  Jesse S. Jin,et al.  Pose determination and viewpoint determination of human head in video conferencing based on head movement , 2004, 10th International Multimedia Modelling Conference, 2004. Proceedings..

[12]  Daniel Rueckert,et al.  Using Photo-Consistency to Register 2D Optical Images of the Human Face to a 3D Surface Model , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Matthew Brand,et al.  Flexible flow for 3D nonrigid tracking and shape recovery , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[14]  Kentaro Toyama,et al.  Gaze-awareness for Videoconferencing: A Software Approach , 2000 .

[15]  Daniel Raviv,et al.  2D feature tracking algorithm for motion analysis , 1995, Pattern Recognit..

[16]  Rama Chellappa,et al.  3D face reconstruction from video using a generic model , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[17]  Songde Ma,et al.  3D face image acquisition and reconstruction system , 1998, IMTC/98 Conference Proceedings. IEEE Instrumentation and Measurement Technology Conference. Where Instrumentation is Going (Cat. No.98CH36222).

[18]  Zhiyong Huang,et al.  Interactive 3D modeling using only one image , 2000, VRST '00.

[19]  S. Lai,et al.  An integrated approach to 3D face model reconstruction from video , 2001, Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems.

[20]  Wolfgang Niem,et al.  Camera viewpoint control for the automatic reconstruction of 3D objects , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[21]  Lisa M. Brown,et al.  A survey of image registration techniques , 1992, CSUR.

[22]  Michael Lindenbaum,et al.  Shape Reconstruction of 3D Bilaterally Symmetric Surfaces , 2000, International Journal of Computer Vision.

[23]  Athinodoros S. Georghiades,et al.  Recovering 3-D Shape and Reflectance From a Small Number of Photographs , 2003, Rendering Techniques.

[24]  Pascal Fua,et al.  3-D Face Modeling from Stereo and Differential Constraints Paradigm , 1998 .

[25]  Qiang Ji,et al.  Real-Time Eye, Gaze, and Face Pose Tracking for Monitoring Driver Vigilance , 2002, Real Time Imaging.