Local Binary Patterns Based Facial Expression Recognition for Efficient Smart Applications

Facial expressions are direct means of communication of human’s emotional state. Hence facial expression recognition (FER) has always been a topic of great interest of researchers specially for smart applications. Numerous approaches have been proposed for FER using Local binary patterns (LBP). This chapter presents an analysis of LBP feature descriptor for FER. State of the art approaches have been discussed that use LBP as their main component. Here, basic LBP operator along with several variants and their main properties are described that have been proved useful for FER. A general framework for FER is described which includes four consecutive modules. These modules are preprocessing, feature extraction, dimensionality reduction and classification. LBP based FER results have been reported on three benchmark datasets JAFFE, CK+ and Yale. Experimentation demonstrates usefulness of LBP in FER.

[1]  M. Sasikumar,et al.  Analysis of Facial Expression using Gabor and , 2009 .

[2]  Yongzhao Zhan,et al.  A neural-AdaBoost based facial expression recognition system , 2014, Expert Syst. Appl..

[3]  Mohamed Elhoseny,et al.  A hybrid model of Internet of Things and cloud computing to manage big data in health services applications , 2018, Future Gener. Comput. Syst..

[4]  Maja Pantic,et al.  Fully Automatic Facial Action Unit Detection and Temporal Analysis , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[5]  Maja Pantic,et al.  Coupled Gaussian processes for pose-invariant facial expression recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Tieniu Tan,et al.  Brief review of invariant texture analysis methods , 2002, Pattern Recognit..

[7]  Matti Pietikäinen,et al.  Gray Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2000, ECCV.

[8]  Markus Turtinen.  Learning and recognizing texture characteristics using local binary patterns , 2007 .

[9]  Marko Heikkilä,et al.  Description of interest regions with local binary patterns , 2009, Pattern Recognit..

[10]  Matti Pietikäinen,et al.  Image Analysis with Local Binary Patterns , 2005, SCIA.

[11]  Nobuyuki Otsu,et al.  Facial expression recognition using Fisher weight maps , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[12]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[13]  Vasif V. Nabiyev,et al.  Down syndrome recognition using local binary patterns and statistical evaluation of the system , 2011, Expert Syst. Appl..

[14]  Swati Nigam,et al.  Multiresolution approach for multiple human detection using moments and local binary patterns , 2014, Multimedia Tools and Applications.

[15]  Zhengyou Zhang,et al.  Feature-Based Facial Expression Recognition: Sensitivity Analysis and Experiments with A Multilayer Perceptron , 1999, Int. J. Pattern Recognit. Artif. Intell..

[16]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[17]  Maja Pantic,et al.  Towards the automatic detection of spontaneous agreement and disagreement based on nonverbal behaviour: A survey of related cues, databases, and tools , 2013, Image Vis. Comput..

[18]  Swati Nigam,et al.  Integration of moment invariants and uniform local binary patterns for human activity recognition in video sequences , 2016, Multimedia Tools and Applications.

[19]  Arko Lucieer,et al.  Segmentation of multispectral high-resolution satellite imagery based on integrated feature distributions , 2010 .

[20]  Maja Pantic,et al.  Web-based database for facial expression analysis , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[21]  Alaa Mohamed Riad,et al.  A machine learning model for improving healthcare services on cloud computing environment , 2018 .

[22]  Emam Hossain,et al.  Person-independent facial expression recognition based on compound local binary pattern (CLBP) , 2014, Int. Arab J. Inf. Technol..

[23]  Matti Pietikäinen,et al.  Rotation-Invariant Image and Video Description With Local Binary Pattern Features , 2012, IEEE Transactions on Image Processing.

[24]  Mohamed Elhoseny,et al.  The impact of the hybrid platform of internet of things and cloud computing on healthcare systems: opportunities, challenges, and open problems , 2017, Journal of Ambient Intelligence and Humanized Computing.

[25]  Ling Shao,et al.  Spatio-temporal Dynamic Texture Descriptors for Human Motion Recognition , 2011, Intelligent Video Event Analysis and Understanding.

[26]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[27]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Jiliu Zhou,et al.  Geometric feature based facial expression recognition using multiclass support vector machines , 2009, 2009 IEEE International Conference on Granular Computing.

[29]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[30]  Mika Laiho,et al.  Towards Understanding the Formation of Uniform Local Binary Patterns , 2013 .

[31]  Li WangDong-Chen He,et al.  Texture classification using texture spectrum , 1990, Pattern Recognit..

[32]  Loris Nanni,et al.  Combining different local binary pattern variants to boost performance , 2011, Expert Syst. Appl..

[33]  Paul F. Whelan,et al.  Image segmentation based on the integration of colour-texture descriptors - A review , 2011, Pattern Recognit..

[34]  Maja Pantic,et al.  Facial Action Unit Detection using Probabilistic Actively Learned Support Vector Machines on Tracked Facial Point Data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[35]  Gwen Littlewort,et al.  Recognizing facial expression: machine learning and application to spontaneous behavior , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[36]  Ahmed Farouk,et al.  Secure Medical Data Transmission Model for IoT-Based Healthcare Systems , 2018, IEEE Access.

[37]  Michael J. Lyons,et al.  Evidence and a computational explanation of cultural differences in facial expression recognition. , 2010, Emotion.

[38]  I. Jolliffe.  Principal Component Analysis , 2002 .

[39]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[40]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Chi-Ho Chan.  Multi-scale local Binary Pattern Histogram for Face Recognition , 2007, ICB.

[42]  A. Treves,et al.  A neural network facial expression recognition system using unsupervised local processing , 2001, ISPA 2001. Proceedings of the 2nd International Symposium on Image and Signal Processing and Analysis. In conjunction with 23rd International Conference on Information Technology Interfaces (IEEE Cat..

[43]  Zhenhua Guo,et al.  Rotation invariant texture classification using LBP variance (LBPV) with global matching , 2010, Pattern Recognit..

[44]  Mohamed Elhoseny,et al.  Secure and Robust Fragile Watermarking Scheme for Medical Images , 2018, IEEE Access.

[45]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[46]  Manasi Patil,et al.  Facial Expression Recognition Based on Image Feature , 2012 .

[47]  Zhenhua Guo,et al.  A Completed Modeling of Local Binary Pattern Operator for Texture Classification , 2010, IEEE Transactions on Image Processing.

[48]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Md. Zia Uddin,et al.  An enhanced independent component-based human facial expression recognition from video , 2009, IEEE Transactions on Consumer Electronics.

[50]  Loris Nanni,et al.  Survey on LBP based texture descriptors for image classification , 2012, Expert Syst. Appl..

[51]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[52]  Shengcai Liao,et al.  Learning Multi-scale Block Local Binary Patterns for Face Recognition , 2007, ICB.

[53]  Matti Pietikäinen,et al.  Rotation-invariant texture classification using feature distributions , 2000, Pattern Recognit..

[54]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .