Facial expression recognition with automatic segmentation of face regions using a fuzzy based classification approach

Abstract This paper proposes a facial expression recognition algorithm that automatically detects the facial image contained in a color picture and segments it in two regions of interest (ROI)—the forehead/eyes and the mouth—which are then divided into non-overlapping N  ×  M blocks. Next, the average of the first element of the cross correlation between 54 Gabor functions and each one of the N  ×  M blocks is estimated to generate a matrix of dimension L  ×  NM , where L is the number of training images. This matrix is then inserted into a principal component analysis (PCA) module for dimensionality reduction. Finally, the resulting matrix is used to generate the feature vectors, which are inserted into the proposed low complexity classifier based on clustering and fuzzy logic techniques. This classifier provides recognition rates close to those provided by other high performance classifiers, but with far less computational complexity. The experimental results show that proposed system achieves a recognition rate of about 97% when the feature vector from only one ROI is used, and that the recognition rate increases to approximately 99% when the feature vectors of both ROIs are used. This result means that the proposed method can achieve an overall recognition rate of approximately 97% even when one of the two ROIs is totally occluded.