Automated facial expression recognition system using neural networks

Abstract This paper proposes an automated facial expression recognition system using neural network classifiers. First, we use the rough contour estimation routine, mathematical morphology, and point contour detection method to extract the precise contours of the eyebrows, eyes, and mouth of a face image. Then we define 30 facial characteristic points to describe the position and shape of these three facial features. Facial expressions can be described by combining different action units, which are specified by the basic muscle movements of a human face. We choose six main action units, composed of facial characteristic point movements, as the input vectors of two different neural network‐based expression classifiers including a radial basis function network and a multilayer perceptron network. Using these two networks, we have obtained recognition rates as high as 92.1% in categorizing the facial expressions neutral, anger, or happiness. Simulation results by the computer demonstrate that computers are capable of extracting high‐level or abstract information like humans