Automatic fetal face detection by locating fetal facial features from 3D ultrasound images for navigating fetoscopic tracheal occlusion surgeries

With the wide clinical application of 3D ultrasound (US) imaging, automatic location of fetal facial features from US volumes for navigating fetoscopic tracheal occlusion (FETO) surgeries becomes possible, which plays an important role in reducing surgical risk. In this paper, we propose a feature-based method to automatically detect 3D fetal face and accurately locate key facial features without any priori knowledge or training data. The candidates of the key facial features, such as the nose, eyes, nose upper bridge and upper lip are detected by analyzing the mean and Gaussian curvatures of the facial surface. Each feature is gradually identified from the candidates by a boosting traversal scheme based on the spatial relations between each feature. In experiments, all key feature points are detected for each case, and thus a detection success rate of 100% is achieved by using 72 3D US images from a test database of 6 fetal faces in the frontal view and any pose within 15° from the frontal view, and the location error 3. 18 ± 0.91 mm of the detected upper lip for all test data is obtained, which can be tolerated by the FETO surgery. Moreover, this system has a high efficiency and can detect all key facial features in about 625 ms on a quad-core 2.60 GHz computer.

[1]  Tieniu Tan,et al.  Combining local features for robust nose location in 3D facial data , 2006, Pattern Recognit. Lett..

[2]  Gaile G. Gordon,et al.  Face recognition based on depth maps and surface curvature , 1991, Optics & Photonics.

[3]  Anuj Srivastava,et al.  Three-Dimensional Face Recognition Using Shapes of Facial Curves , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Patrick J. Flynn,et al.  A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition , 2006, Comput. Vis. Image Underst..

[5]  Anil K. Jain,et al.  Matching 2.5D face scans to 3D models , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Anil K. Jain,et al.  Multimodal Facial Feature Extraction for Automatic 3D Face Recognition , 2005 .

[7]  Hiromi T. Tanaka,et al.  Curvature-based face surface recognition using spherical correlation-principal directions for curved object recognition , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[8]  Thomas Vetter,et al.  Expression invariant 3D face recognition with a Morphable Model , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[9]  M. Tchirikov,et al.  Successful Tracheal Occlusion Using Ultrathin Fetoscopic Equipment Combined with Real-Time Three-Dimensional Ultrasound , 2009, European Surgical Research.

[10]  Hanmin Lee,et al.  A randomized trial of fetal endoscopic tracheal occlusion for severe fetal congenital diaphragmatic hernia. , 2003, The New England journal of medicine.

[11]  Andrea F. Abate,et al.  2D and 3D face recognition: A survey , 2007, Pattern Recognit. Lett..

[12]  Manfredo P. do Carmo,et al.  Differential geometry of curves and surfaces , 1976 .

[13]  K. Nicolaides,et al.  Severe diaphragmatic hernia treated by fetal endoscopic tracheal occlusion , 2009, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[14]  Dirk Vandermeulen,et al.  Objective 3D face recognition: Evolution, approaches and challenges. , 2010, Forensic science international.

[15]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Dorin Comaniciu,et al.  Automatic fetal face detection from ultrasound volumes via learning 3D and 2D information , 2009, CVPR.

[17]  Andrea J. van Doorn,et al.  Surface shape and curvature scales , 1992, Image Vis. Comput..

[18]  Ramesh C. Jain,et al.  Invariant surface characteristics for 3D object recognition in range images , 1985, Comput. Vis. Graph. Image Process..

[19]  Gaile G. Gordon,et al.  Face recognition based on depth and curvature features , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Feng Han,et al.  3D human face recognition using point signature , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[21]  Masakatsu G. Fujie,et al.  Development of 6-DOF wire-driven robotic manipulator for minimally invasive fetal surgery , 2011, 2011 IEEE International Conference on Robotics and Automation.

[22]  Cristina Conde,et al.  3D Facial Normalization with Spin Images and Influence of Range Data Calculation over Face Verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[23]  Raimondo Schettini,et al.  3D face detection using curvature analysis , 2006, Pattern Recognit..

[24]  Marc Acheroy,et al.  Automatic 3D face authentication , 2000, Image Vis. Comput..

[25]  José F. Vélez,et al.  Face recognition using 3D surface extracted descriptors , 2003 .

[26]  Gordon Erlebacher,et al.  A novel technique for face recognition using range imaging , 2003, Seventh International Symposium on Signal Processing and Its Applications, 2003. Proceedings..

[27]  T. Banchoff,et al.  Differential Geometry of Curves and Surfaces , 2010 .