Facial Expression Recognition in the Wild Using FAM, RRDB and Vision Transformer Based Convolutional Neural Networks

Facial expression recognition in the wild is a quite challenge task because facial images captured in natural settings are often affected by various factors such as irregular occlusions, inconsistent face angles and varying light levels. To overcome these factors, we presented a new triple-branch deep neural network model containing facial attention module, residual in residual dense block and vision transformer to deal with facial expression recognition problem. The facial attention module can help backbone network extract more useful features. Residual in residual dense block and vision transformer can further get more detailed features. In addition to the image problems caused by above factors, the ratio of number of images for different expressions in the dataset is also very different. We proposed a new loss function to tackle this uneven ration problem. Experimental results on two benchmark in-the-wild datasets show that our model is indeed helpful for images in the wild. We also created a new expressions dataset called Fairfaceplus, which is built by adding expression categories to the original label on FairFace dataset. The code of our proposed method will be made available on GitHub https://github.com/dreampledge/fairfaceplus.