Reconstructing head models from photographs for individualized 3D‐audio processing

Visual fidelity and interactivity are the main goals in Computer Graphics research, but recently also audio is assuming an important role. Binaural rendering can provide extremely pleasing and realistic three‐dimensional sound, but to achieve best results it's necessary either to measure or to estimate individual Head Related Transfer Function (HRTF). This function is strictly related to the peculiar features of ears and face of the listener. Recent sound scattering simulation techniques can calculate HRTF starting from an accurate 3D model of a human head. Hence, the use of binaural rendering on large scale (i.e. video games, entertainment) could depend on the possibility to produce a sufficiently accurate 3D model of a human head, starting from the smallest possible input. In this paper we present a completely automatic system, which produces a 3D model of a head starting from simple input data (five photos and some key‐points indicated by user). The geometry is generated by extracting information from images and accordingly deforming a 3D dummy to reproduce user head features. The system proves to be fast, automatic, robust and reliable: geometric validation and preliminary assessments show that it can be accurate enough for HRTF calculation.

[1]  B F Katz,et al.  Boundary element method calculation of individual head-related transfer function. I. Rigid model calculation. , 2001, The Journal of the Acoustical Society of America.

[2]  Sam Kwong,et al.  Estimation of Two-Dimensional Frequencies Using Modified Matrix Pencil Method , 2007, IEEE Transactions on Signal Processing.

[3]  Nicola D'Apuzzo.  Human face modeling from multi images , 2000 .

[4]  Hans-Peter Seidel,et al.  Fitting a Morphable Model to 3D Scans of Faces , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[5]  Sylvain Lefebvre,et al.  Instant Sound Scattering , 2007, Rendering Techniques.

[6]  Volkan Atalay,et al.  Delaunay Triangulation based 3D Human Face Modeling from Uncalibrated Images , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[7]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[8]  Nadia Magnenat-Thalmann,et al.  Head Modeling from Pictures and Morphing in 3D with Image Metamorphosis Based on Triangulation , 1998, CAPTECH.

[9]  Wen Gao,et al.  Efficient 3D reconstruction for face recognition , 2005, Pattern Recognit..

[10]  Maic Masuch,et al.  HRTF SIMULATIONS THROUGH ACOUSTIC RAYTRACING , 2006 .

[11]  Hui Chen,et al.  Human Ear Recognition in 3D , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  J. C. Middlebrooks,et al.  Psychophysical customization of directional transfer functions for virtual sound localization. , 2000, The Journal of the Acoustical Society of America.

[13]  V. Ralph Algazi,et al.  An adaptable ellipsoidal head model for the interaural time difference , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[14]  Richard O. Duda,et al.  Structural composition and decomposition of HRTFs , 2001, Proceedings of the 2001 IEEE Workshop on the Applications of Signal Processing to Audio and Acoustics (Cat. No.01TH8575).

[15]  Hui Chen,et al.  Contour Matching for 3D Ear Recognition , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[16]  Yuvi Kahana,et al.  Numerical modelling of the spatial acoustic response of the human pinna , 2006 .

[17]  Frédo Durand,et al.  Texture design using a simplicial complex of morphable textures , 2005, SIGGRAPH '05.

[18]  Paolo Cignoni,et al.  Masked photo blending: Mapping dense photographic data set on high-resolution sampled 3D models , 2008, Comput. Graph..

[19]  F L Wightman,et al.  Localization using nonindividualized head-related transfer functions. , 1993, The Journal of the Acoustical Society of America.

[20]  Durand R. Begault,et al.  3-D Sound for Virtual Reality and Multimedia Cambridge , 1994 .

[21]  Paolo Cignoni,et al.  Metro: Measuring Error on Simplified Surfaces , 1998, Comput. Graph. Forum.

[22]  Markus H. Gross,et al.  Texturing Internal Surfaces from a Few Cross Sections , 2007, Comput. Graph. Forum.

[23]  F. Brian,et al.  INTERNATIONAL CONGRESS ON ACOUSTICS MADRID , 2-7 SEPTEMBER 2007 ROUND ROBIN COMPARISON OF HRTF MEASUREMENT SYSTEMS : PRELIMINARY RESULTS , 2007 .

[24]  E. Jeges,et al.  Model-Based Human Ear Identification , 2006, 2006 World Automation Congress.

[25]  F. Durand,et al.  Texture design using a simplicial complex of morphable textures , 2005, SIGGRAPH 2005.

[26]  Véronique Larcher,et al.  Techniques de spatialisation des sons pour la réalité virtuelle , 2001 .

[27]  WILLIAM G. GARDNER.  Spatial Audio Reproduction: Towards Individualized Binaural Sound , 2004 .

[28]  Volker Blanz,et al.  Face recognition based on a 3D morphable model , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[29]  Tong-Yee Lee,et al.  Photo-realistic 3D Head Modeling Using Multi-view Images , 2004, ICCSA.

[30]  J. Blauert.  Spatial Hearing: The Psychophysics of Human Sound Localization , 1983 .

[31]  Yasuhiro Oue,et al.  Improved 3D head reconstruction system based on combining shape-from-silhouette with two-stage stereo algorithm , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..