A hybrid approach to building face shape classifier for hairstyle recommender system

Abstract Identifying human face shape is the first and the most vital process prior to choosing the right hairstyle to wear on according to guidelines from hairstyle experts, especially for women. This work presents a novel framework for a hairstyle recommender system that is based on face shape classifier. This framework enables an automatic hairstyle recommendation with a single face image. This has a direct impact on beauty industry service providers. It can simulate how the user looks like when she is wearing the chosen hairstyle recommended by the expert system. The model used in this framework is based on Support Vector Machine. The framework is evaluated on hand-crafted, deep-learned (VGG-face) features and VGG-face fine-tuned version for the face shape classification task. In addition to evaluating these individual features by a well-designed framework, we attempted to fuse these three descriptors together in order to improve the performance of the classification task. Two combination techniques were employed, namely: Vector Concatenation and Multiple Kernel Learning (MKL) techniques. All the hyper-parameters of the model were optimised by using Particle Swarm Optimisation. The results show that combining hand-crafted and VGG-face descriptors with MKL yielded the best results at 70.3% of accuracy which was statistically significantly better than using individual features. Thus, combining multiple representations of the data with MKL can improve the overall performance of the expert system. In addition, this proves that hand-crafted descriptor can be complementary to deep-learned descriptor.

[1]  Theekapun Charoenpong,et al.  Face shape classification from 3D human data by using SVM , 2014, The 7th 2014 Biomedical Engineering International Conference.

[2]  Simon Lucey,et al.  Face alignment through subspace constrained mean-shifts , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[3]  Chen Fang,et al.  Visually-Aware Fashion Recommendation and Design with Generative Image Models , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[4]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[5]  Davide Ballabio,et al.  The Kohonen and CP-ANN toolbox: A collection of MATLAB modules for Self Organizing Maps and Counterpropagation Artificial Neural Networks , 2009 .

[6]  Dimitrios I. Fotiadis,et al.  Multiple Kernel Learning Algorithms and Their Use in Biomedical Informatics , 2016 .

[7]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[8]  Kunihiko Fukushima,et al.  Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Visual Pattern Recognition , 1982 .

[9]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[10]  Stefan Winkler,et al.  Deep Learning for Emotion Recognition on Small Datasets using Transfer Learning , 2015, ICMI.

[11]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[12]  Youngjoon Han,et al.  An AAM-based Face Shape Classification Method Used for Facial Expression Recognition , 2013 .

[13]  David Zhang,et al.  Facial beauty analysis based on geometric feature: Toward attractiveness assessment application , 2017, Expert Syst. Appl..

[14]  S. Siegel,et al.  Nonparametric Statistics for the Behavioral Sciences , 2022, The SAGE Encyclopedia of Research Design.

[15]  Erik Learned-Miller,et al.  FDDB: A benchmark for face detection in unconstrained settings , 2010 .

[16]  Alioune Ngom,et al.  A review on machine learning principles for multi-view biological data integration , 2016, Briefings Bioinform..

[17]  Sule Yildirim Yayilgan,et al.  Combining deep learning and hand-crafted features for skin lesion classification , 2016, 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA).

[18]  Wisuwat Sunhem,et al.  An approach to face shape classification for hairstyle recommendation , 2016, 2016 Eighth International Conference on Advanced Computational Intelligence (ICACI).

[19]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[20]  Kitsuchart Pasupa,et al.  Drug screening with Elastic-net multiple kernel learning , 2013, 13th IEEE International Conference on BioInformatics and BioEngineering.

[21]  Kitsuchart Pasupa,et al.  Fashion Finder: A System for Locating Online Stores on Instagram from Product Images , 2018, 2018 10th International Conference on Information Technology and Electrical Engineering (ICITEE).

[22]  Xiang Li,et al.  An enhanced deep feature representation for person re-identification , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[23]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Marko Robnik-Sikonja,et al.  An adaptation of Relief for attribute estimation in regression , 1997, ICML.

[25]  Pichao Wang,et al.  Combining ConvNets with hand-crafted features for action recognition based on an HMM-SVM classifier , 2017, Multimedia Tools and Applications.

[26]  Wisuwat Sunhem,et al.  A comparison between shallow and deep architecture classifiers on small dataset , 2016, 2016 8th International Conference on Information Technology and Electrical Engineering (ICITEE).

[27]  Yang Zhong,et al.  Face attribute prediction using off-the-shelf CNN features , 2016, 2016 International Conference on Biometrics (ICB).

[28]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Zhengpeng Wu,et al.  Elastic Multiple Kernel Learning , 2011 .

[30]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[31]  Mikkel B. Stegmann,et al.  Active appearance models: Theory and cases , 2000 .

[32]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[34]  N K Bansode,et al.  Face Shape Classification Based on Region Similarity, Correlation and Fractal Dimensions , 2016 .

[35]  Qiang Yang,et al.  Distant Domain Transfer Learning , 2017, AAAI.

[36]  Mislav Grgic,et al.  SCface – surveillance cameras face database , 2011, Multimedia Tools and Applications.

[37]  Fengying Wang,et al.  Research on the Personalized Recommendation Algorithm for Hairdressers , 2016 .