What's in a Smile? Initial Results of Multilevel Principal Components Analysis of Facial Shape and Image Texture

Multilevel principal components analysis (mPCA) has previously been shown to provide a simple and straightforward method of forming point distribution models that can be used in (active) shape models. Here we extend the mPCA approach to model image texture as well as shape. As a test case, we consider a set of (2D frontal) facial images from a group of 80 Finnish subjects (34 male; 46 female) with two different facial expressions (smiling and neutral) per subject. Shape (in terms of landmark points) and image texture are considered separately in this initial analysis. Three-level models are constructed that contain levels for biological sex, “within-subject” variation (i.e., facial expression), and “between-subject” variation (i.e., all other sources of variation). By considering eigenvalues, we find that the order of importance as sources of variation for facial shape is: facial expression (47.5%), between-subject variations (45.1%), and then biological sex (7.4%). By contrast, the order for image texture is: between-subject variations (55.5%), facial expression (37.1%), and then biological sex (7.4%). The major modes for the facial expression level of the mPCA models clearly reflect changes in increased mouth size and increased prominence of cheeks during smiling for both shape and texture. Even subtle effects such as changes to eyes and nose shape during smile are seen clearly. The major mode for the biological sex level of the mPCA models similarly relates clearly to changes between male and female. Model fits yield “scores” for each principal component that show strong clustering for both shape and texture by biological sex and facial expression at appropriate levels of the model. We conclude that mPCA correctly decomposes sources of variation due to biological sex and facial expression (etc.) and that it provides a reliable method of forming models of both shape and image texture.

[1]  Bernd Neumann,et al.  Computer Vision — ECCV’98 , 1998, Lecture Notes in Computer Science.

[2]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[3]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  S. Richmond,et al.  A three-dimensional look for facial differences between males and females in a British-Caucasian sample aged 151/2 years old. , 2008, Orthodontics & craniofacial research.

[5]  Timothy F. Cootes,et al.  A mixture model for representing shape variation , 1999, Image Vis. Comput..

[6]  C J Taylor,et al.  Anatomical statistical models and their role in feature extraction. , 2004, The British journal of radiology.

[7]  Reinhilde Jacobs,et al.  Detecting Reduced Bone Mineral Density From Dental Radiographs Using Statistical Shape Models , 2007, IEEE Transactions on Information Technology in Biomedicine.

[8]  Josef Kittler,et al.  Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Timothy F. Cootes,et al.  Statistical models of face images - improving specificity , 1998, Image Vis. Comput..

[10]  Andrew Gelman,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2006 .

[11]  Timothy F. Cootes,et al.  Face Recognition Using Active Appearance Models , 1998, ECCV.

[12]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[13]  Christopher J. Taylor,et al.  Statistical Models of Face Images: Recent Advances , 1996, BMVC.

[14]  J. Velemínská,et al.  Surface facial modelling and allometry in relation to sexual dimorphism. , 2012, Homo : internationale Zeitschrift fur die vergleichende Forschung am Menschen.

[15]  Rama Chellappa,et al.  Discriminant Analysis for Recognition of Human Face Images (Invited Paper) , 1997, AVBPA.

[16]  Alexei I. Zhurov,et al.  Initial Results of Multilevel Principal Components Analysis of Facial Shape , 2017, MIUA.

[17]  Timothy F. Cootes,et al.  Active Shape Models and the shape approximation problem , 1996, Image Vis. Comput..

[18]  Timothy F. Cootes,et al.  Use of active shape models for locating structures in medical images , 1994, Image Vis. Comput..

[19]  Mohammed Benjelloun,et al.  Multilevel statistical shape models: A new framework for modeling hierarchical structures , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[20]  Brian S. Caffo,et al.  Multilevel functional principal component analysis , 2009 .

[21]  A. Lanitis,et al.  Model-based interpretation of complex and variable images. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[22]  H. Popat,et al.  Multilevel principal component analysis (mPCA) in shape analysis: A feasibility study in medical and dental imaging , 2016, Comput. Methods Programs Biomed..