Optical Flow Filtering-Based Micro-Expression Recognition Method

The recognition accuracy of micro-expressions in the field of facial expressions is still understudied, as current research methods mainly focus on feature extraction and classification. Based on optical flow and decision thinking theory, we propose a novel micro-expression recognition method, which can filter low-quality micro-expression video clips. Determined by preset thresholds, we develop two optical flow filtering mechanisms: one based on two-branch decisions (OFF2BD) and the other based on three-way decisions (OFF3WD). In OFF2BD, which use the classical binary logic to classify images, and divide the images into positive or negative domain for further filtering. Differ from the OFF2BD, OFF3WD added boundary domain to delay to judge the motion quality of the images. In this way, the video clips with low degree of morphological change can be eliminated, so as to directly improve the quality of micro-expression features and recognition rate. From the experimental results, we verify the recognition accuracy of 61.57%, and 65.41% for CASMEII, and SMIC datasets, respectively. Through the comparative analysis, it shows that the scheme can effectively improve the recognition performance.

[1]  Nicholas Costen,et al.  SAMM: A Spontaneous Micro-Facial Movement Dataset , 2018, IEEE Transactions on Affective Computing.

[2]  Kidiyo Kpalma,et al.  Motion descriptors for micro-expression recognition , 2018, Signal Process. Image Commun..

[3]  Guoying Zhao,et al.  Micro-Expression Recognition Using Color Spaces , 2015, IEEE Transactions on Image Processing.

[4]  Ming Yu,et al.  Micro-expression recognition based on CBP-TOP feature with ELM , 2015 .

[5]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[6]  Gangqiang Zhang,et al.  A three-way decision method in a hybrid decision information system and its application in medical diagnosis , 2020, Artificial Intelligence Review.

[7]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[8]  David Cristinacce,et al.  Automatic feature localisation with constrained local models , 2008, Pattern Recognit..

[9]  Matti Pietikäinen,et al.  Spontaneous facial micro-expression analysis using Spatiotemporal Completed Local Quantized Patterns , 2016, Neurocomputing.

[10]  Matti Pietikäinen,et al.  Extended Local Binary Patterns for Efficient and Robust Spontaneous Facial Micro-Expression Recognition , 2019, IEEE Access.

[11]  Zhifei Zhang,et al.  A three-way decisions model with probabilistic rough sets for stream computing , 2017, Int. J. Approx. Reason..

[12]  P. Ekman,et al.  Nonverbal Leakage and Clues to Deception †. , 1969, Psychiatry.

[13]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Timothy F. Cootes,et al.  Active Shape Models and the shape approximation problem , 1996, Image Vis. Comput..

[15]  Yu-Hsin Chen,et al.  Measuring dynamic micro-expressions via feature extraction methods , 2017, J. Comput. Sci..

[16]  Sujing Wang,et al.  Micro-expression recognition with small sample size by transferring long-term convolutional neural network , 2018, Neurocomputing.

[17]  Cheonshik Kim,et al.  A Light-Weight Practical Framework for Feces Detection and Trait Recognition , 2020, Sensors.

[18]  Xiaolan Fu,et al.  CAS(ME)$^2$ : A Database for Spontaneous Macro-Expression and Micro-Expression Spotting and Recognition , 2018, IEEE Transactions on Affective Computing.

[19]  Guoying Zhao,et al.  CASME II: An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation , 2014, PloS one.

[20]  Wei Liang,et al.  Optimization of Basic Clustering for Ensemble Clustering: An Information-Theoretic Perspective , 2019, IEEE Access.

[21]  Lei Wang,et al.  A weighted feature extraction method based on temporal accumulation of optical flow for micro-expression recognition , 2019, Signal Process. Image Commun..

[22]  KokSheik Wong,et al.  Less is More: Micro-expression Recognition from Video using Apex Frame , 2016, Signal Process. Image Commun..

[23]  Feng Xu,et al.  Microexpression Identification and Categorization Using a Facial Dynamics Map , 2017, IEEE Transactions on Affective Computing.

[24]  Steven C. H. Hoi,et al.  Face Detection using Deep Learning: An Improved Faster RCNN Approach , 2017, Neurocomputing.

[25]  Huai-Qian Khor,et al.  Revealing the Invisible With Model and Data Shrinking for Composite-Database Micro-Expression Recognition , 2020, IEEE Transactions on Image Processing.

[26]  Ming Wan,et al.  Combining 3D Convolutional Neural Networks with Transfer Learning by Supervised Pre-Training for Facial Micro-Expression Recognition , 2019, IEICE Trans. Inf. Syst..

[27]  Gwen Littlewort,et al.  Toward Practical Smile Detection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Kidiyo Kpalma,et al.  Macro-to-micro transformation model for micro-expression recognition , 2017, J. Comput. Sci..

[29]  Guoying Zhao,et al.  Spatiotemporal Recurrent Convolutional Networks for Recognizing Spontaneous Micro-Expressions , 2019, IEEE Transactions on Multimedia.

[30]  Wang,et al.  Real-Time Pre-Identification and Cascaded Detection for Tiny Faces , 2019, Applied Sciences.

[31]  Lu Leng,et al.  StoolNet for Color Classification of Stool Medical Images , 2019, Electronics.

[32]  Aurobinda Routray,et al.  Fuzzy Histogram of Optical Flow Orientations for Micro-Expression Recognition , 2019, IEEE Transactions on Affective Computing.

[33]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..