Extreme Learning Machine Ensemble Using Bagging for Facial Expression Recognition

Abstract —An extreme learning machine (ELM) is a recently proposed learning algorithm for a single-layer feed forward neural network. In this paper we studied the ensemble of ELM by using a bagging algorithm for facial expression recognition (FER). Facial expression analysis is widely used in the behavior interpretation of emotions, for cognitive science, and social interactions. This paper presents a method for FER based on the histogram of orientation gradient (HOG) features using an ELM ensemble. First, the HOG features were extracted from the face image by dividing it into a number of small cells. A bagging algorithm was then used to construct many different bags of training data and each of them was trained by using separate ELMs. To recognize the expression of the input face image, HOG features were fed to each trained ELM and the results were combined by using a majority voting scheme. The ELM ensemble using bagging improves the generalized capability of the network significantly. The two available datasets (JAFFE and CK+) of facial expressions were used to evaluate the performance of the proposed classification system. Even the performance of individual ELM was smaller and the ELM ensemble using a bagging algorithm improved the recognition performance significantly.

[1]  Chia-Feng Juang,et al.  Using self-organizing fuzzy network with support vector learning for face detection in color images , 2008, Neurocomputing.

[2]  Guang-Bin Huang,et al.  Face recognition based on extreme learning machine , 2011, Neurocomputing.

[3]  Ioannis Pitas,et al.  Discriminant Graph Structures for Facial Expression Recognition , 2008, IEEE Transactions on Multimedia.

[4]  Le Hoang Thai,et al.  A Facial Expression Classification System Integrating Canny, Principal Component Analysis and Artificial Neural Network , 2011, ArXiv.

[5]  Deepak Ghimire,et al.  A Robust Face Detection Method Based on Skin Color and Edges , 2013, J. Inf. Process. Syst..

[6]  Oksam Chae,et al.  Robust Facial Expression Recognition Based on Local Directional Pattern , 2010 .

[7]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Shiqing Zhang,et al.  Robust Facial Expression Recognition via Compressive Sensing , 2012, Sensors.

[9]  Maja Pantic,et al.  Meta-Analysis of the First Facial Expression Recognition Challenge , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  Junchul Chun,et al.  Classification of Facial Expression Using SVM for Emotion Care Service System , 2008, 2008 Ninth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing.

[11]  Berkman Sahiner,et al.  Dual system approach to computer-aided detection of breast masses on mammograms. , 2006, Medical physics.

[12]  Zhi-Zhong Mao,et al.  An Ensemble ELM Based on Modified AdaBoost.RT Algorithm for Predicting the Temperature of Molten Steel in Ladle Furnace , 2010, IEEE Transactions on Automation Science and Engineering.

[13]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1990, COLT '90.

[14]  Sujing Wang,et al.  Face recognition using second-order discriminant tensor subspace analysis , 2011, Neurocomputing.

[15]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[16]  Deepak Ghimire,et al.  Automatic facial expression recognition based on features extracted from tracking of facial landmarks , 2014, International Conference on Graphic and Image Processing.

[17]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[18]  Amaury Lendasse,et al.  Adaptive Ensemble Models of Extreme Learning Machines for Time Series Prediction , 2009, ICANN.

[19]  P. Ekman,et al.  Strong evidence for universals in facial expressions: a reply to Russell's mistaken critique. , 1994, Psychological bulletin.

[20]  C. Chibelushi,et al.  Facial Expression Recognition : A Brief Tutorial Overview , 2022 .

[21]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[22]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[23]  Deepak Ghimire,et al.  Geometric Feature-Based Facial Expression Recognition in Image Sequences Using Multi-Class AdaBoost and Support Vector Machines , 2013, Sensors.

[24]  Han Wang,et al.  Ensemble Based Extreme Learning Machine , 2010, IEEE Signal Processing Letters.

[25]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[26]  Ana Belén Moreno,et al.  Differential optical flow applied to automatic facial expression recognition , 2011, Neurocomputing.

[27]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[28]  Yuan Lan,et al.  Ensemble of online sequential extreme learning machine , 2009, Neurocomputing.

[29]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Gwen Littlewort,et al.  Dynamics of Facial Expression Extracted Automatically from Video , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[31]  A. Rogier.  [Communication without words]. , 1971, Tijdschrift voor ziekenverpleging.

[32]  Ashok Samal,et al.  Automatic recognition and analysis of human faces and facial expressions: a survey , 1992, Pattern Recognit..

[33]  Ioannis Pitas,et al.  Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines , 2007, IEEE Transactions on Image Processing.

[34]  Yu Liu,et al.  Simple Ensemble of Extreme Learning Machine , 2009, 2009 2nd International Congress on Image and Signal Processing.

[35]  Michael J. Lyons,et al.  Automatic Classification of Single Facial Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[37]  Robert E. Schapire,et al.  The strength of weak learnability , 1990, Mach. Learn..