An automatic approach for classification and categorisation of lip morphological traits

Classification of facial traits (e.g., lip shape) is an important area of medical research, for example, in determining associations between lip traits and genetic variants which may lead to a cleft lip. In clinical situations, classification of facial traits is usually performed subjectively directly on the individual or recorded later from a three-dimensional image, which is time consuming and prone to operator errors. The present study proposes, for the first time, an automatic approach for the classification and categorisation of lip area traits. Our approach uses novel three-dimensional geometric features based on surface curvatures measured along geodesic paths between anthropometric landmarks. Different combinations of geodesic features are analysed and compared. The effect of automatically identified categories on the face is visualised using a partial least squares method. The method was applied to the classification and categorisation of six lip shape traits (philtrum, Cupid’s bow, lip contours, lip-chin, and lower lip tone) in a large sample of 4747 faces of normal British Western European descents. The proposed method demonstrates correct automatic classification rate of up to 90%.

[1]  Peter Hammond,et al.  Facial Dysmorphism Across the Fetal Alcohol , 2016 .

[2]  Ajmal S. Mian,et al.  Perceptual Differences between Men and Women: A 3D Facial Morphometric Perspective , 2014, 2014 22nd International Conference on Pattern Recognition.

[3]  Tshilidzi Marwala,et al.  Image Classification Using SVMs: One-against-One Vs One-against-All , 2007, ArXiv.

[4]  S Richmond,et al.  Reproducibility of facial soft tissue landmarks on 3D laser-scanned facial images. , 2009, Orthodontics & craniofacial research.

[5]  Lutz Funk,et al.  Nanostructured medical sutures with antibacterial properties. , 2015, Biomaterials.

[6]  Said Safi,et al.  Facial Surface Analysis using Iso-Geodesic Curves in Three Dimensional Face Recognition System , 2016, ArXiv.

[7]  Hermann Ney,et al.  Deep Learning of Mouth Shapes for Sign Language , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[8]  A. J. Collins,et al.  Introduction To Multivariate Analysis , 1981 .

[9]  Mark Meyer,et al.  Discrete Differential-Geometry Operators for Triangulated 2-Manifolds , 2002, VisMath.

[10]  Driss Aboutajdine,et al.  Boosting 3-D-Geometric Features for Efficient Face Recognition and Gender Classification , 2012, IEEE Transactions on Information Forensics and Security.

[11]  Alfred O. Hero,et al.  Clustering with a new distance measure based on a dual-rooted tree , 2013, Inf. Sci..

[12]  K. Devriendt,et al.  Elements of morphology: Standard terminology for the lips, mouth, and oral region , 2009, American journal of medical genetics. Part A.

[13]  Mohammed Bennamoun,et al.  A Comprehensive Performance Evaluation of 3D Local Feature Descriptors , 2015, International Journal of Computer Vision.

[14]  W. P. Koppen Learning 3D Face Shape Features From Local Coherence. , 2014 .

[15]  A. Toma Characterization of normal facial features and their association with genes , 2014 .

[16]  Sung Won Ko,et al.  Development of polyamide-6,6/chitosan electrospun hybrid nanofibrous scaffolds for tissue engineering application. , 2016, Carbohydrate polymers.

[17]  Eréndira Rendón,et al.  A comparison of internal and external cluster validation indexes , 2011 .

[18]  Chiarella Sforza,et al.  Age- and sex-related changes in three-dimensional lip morphology. , 2010, Forensic science international.

[19]  James Algina,et al.  Conducting Power Analyses for Anova and Ancova in between-Subjects Designs , 2003, Evaluation & the health professions.

[20]  Alexander M. Bronstein,et al.  Efficient Computation of Isometry-Invariant Distances Between Surfaces , 2006, SIAM J. Sci. Comput..

[21]  Kemal Polat,et al.  Brain MRI Segmentation based on Different Clustering Algorithms , 2016 .

[22]  Eréndira Rendón,et al.  Internal versus External cluster validation indexes , 2011 .

[23]  L L Merrifield,et al.  The profile line as an aid in critically evaluating facial esthetics. , 1966, American journal of orthodontics.

[24]  P. Claes,et al.  Improved facial outcome assessment using a 3D anthropometric mask. , 2012, International journal of oral and maxillofacial surgery.

[25]  Matthieu Cord,et al.  Bag-of-Words Image Representation: Key Ideas and Further Insight , 2014, Fusion in Computer Vision.

[26]  Zhongke Wu,et al.  3D Facial Similarity Measure Based on Geodesic Network and Curvatures , 2014 .

[27]  A. Young,et al.  Understanding face recognition. , 1986, British journal of psychology.

[28]  Alan C. Bovik,et al.  Anthropometric 3D Face Recognition , 2010, International Journal of Computer Vision.

[29]  Shi-Qing Xin,et al.  Improving Chen and Han's algorithm on the discrete geodesic problem , 2009, TOGS.

[30]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[31]  A. O'Toole,et al.  Sex Classification is Better with Three-Dimensional Head Structure Than with Image Intensity Information , 1997, Perception.

[32]  G. Shelke,et al.  Face Recognition and Gender Classification Using Features of Lips , 2014 .

[33]  Pierre Legendre,et al.  An empirical comparison of permutation methods for tests of partial regression coefficients in a linear model , 1999 .

[34]  Christophe Mues,et al.  An experimental comparison of classification algorithms for imbalanced credit scoring data sets , 2012, Expert Syst. Appl..

[35]  Steven J. Gortler,et al.  Fast exact and approximate geodesics on meshes , 2005, ACM Trans. Graph..

[36]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[37]  Alexander M. Bronstein,et al.  Parallel algorithms for approximation of distance maps on parametric surfaces , 2008, TOGS.

[38]  Tatsuo Nakajima,et al.  Analysis of 109 Japanese children's lip and nose shapes using 3-dimensional digitizer. , 2005, British journal of plastic surgery.

[39]  J. Mejino,et al.  Computer processable classification of craniofacial clefts , 2015, Front. Physiol..

[40]  Chung How Kau,et al.  Three-dimensional analysis of facial morphology surface changes in untreated children from 12 to 14 years of age. , 2008, American journal of orthodontics and dentofacial orthopedics : official publication of the American Association of Orthodontists, its constituent societies, and the American Board of Orthodontics.

[41]  Paul J. Besl,et al.  Method for registration of 3-D shapes , 1992, Other Conferences.

[42]  S. Richmond,et al.  The prevalence of lip vermilion morphological traits in a 15‐year‐old population , 2013, American journal of medical genetics. Part A.

[43]  S. Richmond,et al.  Genetic and Environmental Contributions to Facial Morphological Variation: A 3D Population-Based Twin Study , 2016, PloS one.

[44]  Max Kuhn,et al.  Applied Predictive Modeling , 2013 .

[45]  Linda G. Shapiro,et al.  Shape-based Classification of 3d Facial Data to Support 22q11.2ds Craniofacial Research , 2022 .

[46]  Hassen Drira,et al.  Pose and Expression-Invariant 3D Face Recognition using Elastic Radial Curves , 2010, BMVC.

[47]  J. Fu,et al.  Convergence of curvatures in secant approximations , 1993 .

[48]  R. A. Armstrong,et al.  An introduction to analysis of variance (ANOVA) with special reference to data from clinical experiments in optometry , 2000, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[49]  Anil K. Jain,et al.  Matching 2.5D face scans to 3D models , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Ethem Alpaydin,et al.  Introduction to machine learning , 2004, Adaptive computation and machine learning.

[51]  Liming Chen,et al.  A coarse-to-fine curvature analysis-based rotation invariant 3D face landmarking , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[52]  B Solow,et al.  Natural head position in standing subjects. , 1971, Acta odontologica Scandinavica.

[53]  Federica Marcolin,et al.  3D human face description: landmarks measures and geometrical features , 2012, Image Vis. Comput..

[54]  Michael G. Strintzis,et al.  3-D Face Recognition With the Geodesic Polar Representation , 2007, IEEE Transactions on Information Forensics and Security.

[55]  T. Beaty,et al.  Confirming genes influencing risk to cleft lip with/without cleft palate in a case–parent trio study , 2013, Human Genetics.

[56]  D. Lawlor,et al.  Cohort Profile: The ‘Children of the 90s’—the index offspring of the Avon Longitudinal Study of Parents and Children , 2012, International journal of epidemiology.

[57]  J. Illingworth,et al.  Automatic landmarking of faces in 3D - ALF 3D , 2008 .

[58]  R A Holdaway,et al.  A soft-tissue cephalometric analysis and its use in orthodontic treatment planning. Part I. , 1983, American journal of orthodontics.

[59]  Sahibsingh A. Dudani The Distance-Weighted k-Nearest-Neighbor Rule , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[60]  A. Ben Hamza,et al.  Geodesic matching of triangulated surfaces , 2006, IEEE Transactions on Image Processing.

[61]  Pierre Alliez,et al.  Anisotropic polygonal remeshing , 2003, ACM Trans. Graph..

[62]  Alexei I. Zhurov,et al.  A 3D morphometric perspective for facial gender analysis and classification using geodesic path curvature features , 2018, Computational Visual Media.

[63]  Federico Tombari,et al.  Unique Signatures of Histograms for Local Surface Description , 2010, ECCV.

[64]  C. Grimm,et al.  Improved Curvature Estimation on Triangular Meshes , 2004 .

[65]  Syed Zulqarnain Gilani,et al.  Geometric Facial Gender Scoring: Objectivity of Perception , 2014, PloS one.

[66]  A. Bovik,et al.  Three Dimensional Face Recognition Using Iso-Geodesic and Iso-Depth Curves , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[67]  Yi Chen,et al.  A preliminary study on the key factors contributing to the attractive lips of Chinese children. , 2012, Asian Pacific journal of tropical medicine.

[68]  A. M. Burton,et al.  What's the Difference between Men and Women? Evidence from Facial Measurement , 1993, Perception.

[69]  P. Claes,et al.  Spatially dense morphometrics of craniofacial sexual dimorphism in 1‐year‐olds , 2016, Journal of anatomy.

[70]  Alexei I. Zhurov,et al.  Averaging Facial Images , 2010 .

[71]  Anil K. Jain,et al.  Automatic feature extraction for multiview 3D face recognition , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[72]  Fouad Khan,et al.  An initial seed selection algorithm for k-means clustering of georeferenced data to improve replicability of cluster assignments for mapping application , 2016, Appl. Soft Comput..

[73]  R M Ricketts,et al.  Esthetics, environment, and the law of lip relation. , 1968, American journal of orthodontics.

[74]  Peter Hammond,et al.  The face signature of fibrodysplasia ossificans progressiva , 2012, American journal of medical genetics. Part A.

[75]  Victoria Interrante,et al.  A novel cubic-order algorithm for approximating principal direction vectors , 2004, TOGS.

[76]  Alex M. Andrew,et al.  Level Set Methods and Fast Marching Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Materials Science (2nd edition) , 2000 .

[77]  C J Burstone,et al.  Lip posture and its significance in treatment planning. , 1967, American journal of orthodontics.

[78]  C. Sforza,et al.  Three-dimensional analysis of labial morphology: effect of sex and age. , 2011, International journal of oral and maxillofacial surgery.

[79]  Hui Xiong,et al.  Understanding of Internal Clustering Validation Measures , 2010, 2010 IEEE International Conference on Data Mining.

[80]  K. P. N. V. Satyasree,et al.  An Exhaustive Literature Review on Class Imbalance Problem , 2013 .

[81]  Christian Rössl,et al.  Laplacian surface editing , 2004, SGP '04.

[82]  P. Manikandan,et al.  IMBALANCED DATASET CLASSIFICATION AND SOLUTIONS : A REVIEW , 2014 .

[83]  Dae Joong Kim,et al.  Musculature of the Pars Marginalis of the Upper Orbicularis Oris Muscle , 2007, The Journal of craniofacial surgery.

[84]  M. Markey,et al.  Quantitative Anthropometric Measures of Facial Appearance of Healthy Hispanic/Latino White Children: Establishing Reference Data for Care of Cleft Lip With or Without Cleft Palate , 2017 .

[85]  Yaxin Bi,et al.  Sentiment Analysis of Customer Reviews: Balanced versus Unbalanced Datasets , 2011, KES.

[86]  Peter Hammond,et al.  Facial Dysmorphism Across the Fetal Alcohol Spectrum , 2013, Pediatrics.

[87]  David Dunaway,et al.  An Automatic 3D Facial Landmarking Algorithm Using 2D Gabor Wavelets , 2016, IEEE Transactions on Image Processing.

[88]  Paul Suetens,et al.  A spatially-dense regression study of facial form and tissue depth: towards an interactive tool for craniofacial reconstruction. , 2014, Forensic science international.

[89]  Darryl Stewart,et al.  Gender classification via lips: static and dynamic features , 2013, IET Biom..

[90]  Ioannis A. Kakadiaris,et al.  Automatic 3D Facial Region Retrieval from Multi-pose Facial Datasets , 2009, 3DOR@Eurographics.

[91]  Mohamed Abouelenien,et al.  A multi-class boosting method for learning from imbalanced data , 2016, Int. J. Granul. Comput. Rough Sets Intell. Syst..

[92]  David Bommes,et al.  Accurate Computation of Geodesic Distance Fields for Polygonal Curves on Triangle Meshes , 2007, VMV.

[93]  Fan,et al.  An automatic 3 D facial landmarking algorithm using 2 D Gabor Wavelets , 2015 .

[94]  Paul L. Rosin,et al.  The assessment of facial variation in 4747 British school children. , 2012, European Journal of Orthodontics.

[95]  Takeshi Saitoh,et al.  Japanese Phone Recognition Using Lip Image Information , 2007, MVA.

[96]  Rich Caruana,et al.  An empirical comparison of supervised learning algorithms , 2006, ICML.

[97]  A. David Marshall,et al.  Automatic Classification of Facial Morphology for Medical Applications , 2015, KES.

[98]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[99]  D. Cramer,et al.  Introduction to Statistics in Psychology , 2005 .