Facial landmark detection and tracking with dynamically adaptive matched filters

Abstract. A reliable method for the detection and tracking of facial landmarks in image sequences is presented. Given a set of prespecified facial landmarks in a reference face image, a bank of composite matched filters is constructed for reliable detection and accurate location of the landmarks in an input image sequence. The filter bank is dynamically adapted to each captured frame by learning from current and past landmark detections and considering geometrical modifications of the landmarks. The detected landmarks are accurately tracked using a kinematic motion model that predicts their coordinates in future frames. The performance of facial landmark detection and tracking obtained with the proposed method is tested by processing real-life image sequences. The obtained results are analyzed and discussed in terms of objective measures.

[1]  Haijun Wang,et al.  Robust and fast object tracking via co-trained adaptive correlation filter , 2019, Optik.

[2]  Ioannis A. Kakadiaris,et al.  3D-2D face recognition with pose and illumination normalization , 2017, Comput. Vis. Image Underst..

[3]  Bahram Javidi,et al.  Distortion-tolerant minimum-mean-squared-error filter for detecting noisy targets in environmental degradation , 2000 .

[4]  Bahram Javidi,et al.  Design of filters to detect a noisy target in nonoverlapping background noise , 1994 .

[5]  Michel F. Valstar,et al.  Cascaded regression with sparsified feature covariance matrix for facial landmark detection , 2016, Pattern Recognit. Lett..

[6]  LI X.RONG,et al.  Survey of maneuvering target tracking. Part I. Dynamic models , 2003 .

[7]  B. V. Vijaya Kumar,et al.  Unconstrained correlation filters. , 1994, Applied optics.

[8]  Zhongmin Wang,et al.  Long-term visual tracking based on adaptive correlation filters , 2018, J. Electronic Imaging.

[9]  D Casasent,et al.  Multivariant technique for multiclass pattern recognition. , 1980, Applied optics.

[10]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[11]  Zheng Zhang,et al.  Pose-invariant face recognition using facial landmarks and Weber local descriptor , 2015, Knowl. Based Syst..

[12]  Victor H. Diaz-Ramirez,et al.  Target tracking with dynamically adaptive correlation , 2016 .

[13]  Xiaodong Gu,et al.  Accurate mask-based spatially regularized correlation filter for visual tracking , 2017, J. Electronic Imaging.

[14]  Jian Sun,et al.  Face Alignment at 3000 FPS via Regressing Local Binary Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Asit K. Datta,et al.  An illumination tolerant class specific 2D subspace based face recognition technique using optimum correlation filter , 2013 .

[16]  Yong-Hwan Lee,et al.  Facial landmarks detection using improved active shape model on android platform , 2013, Multimedia Tools and Applications.

[17]  B V Kumar,et al.  Tutorial survey of composite filter designs for optical correlators. , 1992, Applied optics.

[18]  Ioannis A. Kakadiaris,et al.  Feature fusion for facial landmark detection , 2014, Pattern Recognit..

[19]  B. V. K. Vijaya Kumar,et al.  Maximum Margin Correlation Filter: A New Approach for Localization and Classification , 2013, IEEE Transactions on Image Processing.

[20]  Mário Marques Fernandes,et al.  ADVANCES IN FACE DETECTION AND FACIAL IMAGE ANALYSIS , 2018 .

[21]  D. Casasent,et al.  Correlation synthetic discriminant functions. , 1986, Applied optics.

[22]  Vitaly Kober,et al.  Design of correlation filters for recognition of linearly distorted objects in linearly degraded scenes. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[23]  Vitaly Kober,et al.  Real-time tracking of multiple objects using adaptive correlation filters with complex constraints , 2013 .

[24]  B. Kumar,et al.  Performance measures for correlation filters. , 1990, Applied optics.

[25]  F. Rohlf,et al.  Extensions of the Procrustes Method for the Optimal Superimposition of Landmarks , 1990 .

[26]  Václav Hlavác,et al.  Real-time multi-view facial landmark detector learned by the structured output SVM , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[27]  Rigoberto Juarez-Salazar,et al.  Adaptive matched filter for implicit-target recognition: application in three-dimensional reconstruction. , 2019, Applied optics.

[28]  Vitaly Kober,et al.  Pattern recognition with an adaptive joint transform correlator. , 2006, Applied optics.

[29]  Filiz Bunyak,et al.  iFER: facial expression recognition using automatically selected geometric eye and eyebrow features , 2018, J. Electronic Imaging.

[30]  Steve Serati,et al.  Optical correlator based target detection, recognition, classification, and tracking. , 2012, Applied optics.

[31]  L. P. Yaroslavsky,et al.  III The Theory of Optimal Methods for Localization of Objects in Pictures , 1993 .