Computational face reader based on facial attribute estimation

Chinese face reading has demonstrated the often satisfying capabilities to tell the characteristics (mostly exaggerated as fortune) of a person by reading his/her face, i.e. understanding the fine-grained facial attributes (e.g., length of nose, single/double-fold eyelid, density of eyebrows, etc.). Thus, a smart face reading system should estimate the fine-grained facial attributes well. Therefore, In this paper, we first study the fine-grained facial attribute estimation problem and propose a novel deep convolutional network equipped with a new facial region pooling layer (called FRP-net), to accurately estimate the fine-grained facial attributes. To capture the characteristics of fine-grained facial attributes, the embedded FRP layer implements the pooling operation on the searched facial region windows (locates the region of each facial attribute) instead of the commonly-used sliding windows. Further, we push the proposed fine-grained facial attribute estimation method into the face reading problem and present a computational face reader system to automatically infer the characteristics of a person based on his/her face. For example, it can estimate the attractive and easy-going characteristics of a Chinese person from his/her big eyes according to the Chinese anthroposcopy literature. The experimental results on facial attribute estimation demonstrate the superiority of the proposed FRP-net compared to the baselines, and the qualitative and quantitative evaluations on face reading validate the excellence of the presented face reader system.

[1]  Antonio Torralba,et al.  Modifying the Memorability of Face Photographs , 2013, 2013 IEEE International Conference on Computer Vision.

[2]  Bing Li,et al.  Gender classification by combining clothing, hair and facial component classifiers , 2012, Neurocomputing.

[3]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[5]  Philip H. S. Torr,et al.  BING: Binarized normed gradients for objectness estimation at 300fps , 2019, Computational Visual Media.

[6]  Michael S. Lew,et al.  Deep learning for visual understanding: A review , 2016, Neurocomputing.

[7]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[8]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[9]  Xiaogang Wang,et al.  DeepID3: Face Recognition with Very Deep Neural Networks , 2015, ArXiv.

[10]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[11]  Yan-Ying Chen,et al.  Scalable Face Image Retrieval Using Attribute-Enhanced Sparse Codewords , 2013, IEEE Transactions on Multimedia.

[12]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[13]  Shree K. Nayar,et al.  Attribute and simile classifiers for face verification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[14]  Jian Dong,et al.  Deep domain adaptation for describing people based on fine-grained clothing attributes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Peter N. Belhumeur,et al.  POOF: Part-Based One-vs.-One Features for Fine-Grained Categorization, Face Verification, and Attribute Estimation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Subhransu Maji,et al.  Describing people: A poselet-based approach to attribute classification , 2011, 2011 International Conference on Computer Vision.

[17]  Yang Yang,et al.  Face image classification by pooling raw features , 2014, Pattern Recognit..

[18]  David J. Kriegman,et al.  Two faces are better than one: Face recognition in group photographs , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[19]  Hua Xu,et al.  Questionnaires-based skin attribute prediction using Elman neural network , 2011, Neurocomputing.

[20]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[21]  C. Lawrence Zitnick,et al.  Edge Boxes: Locating Object Proposals from Edges , 2014, ECCV.

[22]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[23]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[24]  Xuelong Li,et al.  Visual Coding in a Semantic Hierarchy , 2015, ACM Multimedia.

[25]  Shuicheng Yan,et al.  Computational Face Reader , 2016, MMM.

[26]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[28]  Shiguang Shan,et al.  AU-inspired Deep Networks for Facial Expression Feature Learning , 2015, Neurocomputing.

[29]  Shree K. Nayar,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence Describable Visual Attributes for Face Verification and Image Search , 2022 .

[30]  Hanjiang Lai,et al.  Personalized Age Progression with Aging Dictionary , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[31]  Yang Yang,et al.  Multitask Spectral Clustering by Exploring Intertask Correlation , 2015, IEEE Transactions on Cybernetics.

[32]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[33]  Kristen Grauman,et al.  Relative attributes , 2011, 2011 International Conference on Computer Vision.

[34]  Hanjiang Lai,et al.  Kinship-Guided Age Progression , 2016, Pattern Recognit..

[35]  Donghoon Lee,et al.  Face attribute classification using attribute-aware correlation map and gated convolutional neural networks , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[36]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[37]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[38]  Donghoon Lee,et al.  Deep Attribute Networks , 2012, ArXiv.

[39]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Xiaogang Wang,et al.  A Deep Sum-Product Architecture for Robust Facial Attributes Analysis , 2013, 2013 IEEE International Conference on Computer Vision.

[41]  C. V. Jawahar,et al.  Cats and dogs , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Erich Elsen,et al.  Deep Speech: Scaling up end-to-end speech recognition , 2014, ArXiv.

[43]  Andrew Zisserman,et al.  Automated Flower Classification over a Large Number of Classes , 2008, 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing.

[44]  Shiguang Shan,et al.  Shape driven kernel adaptation in Convolutional Neural Network for robust facial trait recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[46]  Xiaoyang Tan,et al.  Exploiting relationship between attributes for improved face verification , 2014, Comput. Vis. Image Underst..

[47]  Yue Gao,et al.  Exploiting Web Images for Semantic Video Indexing Via Robust Sample-Specific Loss , 2014, IEEE Transactions on Multimedia.