Landmark Fitting for Sequential Faces Based on Active Shape Model and Tracking Correction

In this paper, a method of landmark fitting for sequential faces is presented based on active shape model(ASM) and tracking correction. This method overcomes the loss of consecutive information between frames, and makes full use of the motion variation information of video sequences in time and space dimensions. Firstly, the optical flow values of several key points on the face are calculated by the large displacement optical flow model. Secondly, the positions of these points in the current frame are located to modify the global shape model of ASM and conduct the precise localization of landmarks in landmark searching. Finally, the rationality of landmark is suppressed to obtain the ultimate results. Our proposed method observably improves the accurate localization of ASM for deformed faces, and takes full advantage of the continuity among video sequences, so that it has significant effect on landmark fitting for sequential faces. Compared with ASM, extensive experiments show that our method performs outstandingly in terms of accuracy and robustness.

[1]  Fred Nicolls,et al.  Locating Facial Features with an Extended Active Shape Model , 2008, ECCV.

[2]  Fanrang Kong,et al.  Facial feature point extraction using a new improved Active Shape Model , 2010, 2010 3rd International Congress on Image and Signal Processing.

[3]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Fred Nicolls,et al.  Active shape models with SIFT descriptors and MARS , 2015, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[5]  Lei Huang,et al.  Initialization and Pose Alignment in Active Shape Model , 2010, 2010 20th International Conference on Pattern Recognition.

[6]  Stephen Milborrow.  The MUCT Landmarked Face Database , 2010 .

[7]  Bülent Sankur,et al.  A comparative study of face landmarking techniques , 2013, EURASIP J. Image Video Process..

[8]  Kurt Keutzer,et al.  Dense Point Trajectories by GPU-Accelerated Large Displacement Optical Flow , 2010, ECCV.

[9]  Jian Sun,et al.  Face Alignment at 3000 FPS via Regressing Local Binary Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Luc Van Gool,et al.  Real time 3D face alignment with Random Forests-based Active Appearance Models , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[11]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[12]  Heng Yang,et al.  Facial feature point detection: A comprehensive survey , 2014, Neurocomputing.

[13]  Tom E. Bishop,et al.  Multiview Active Shape Models with SIFT Descriptors for the 300-W Face Landmark Challenge , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[14]  Takeo Kanade,et al.  Dense 3D face alignment from 2D video for real-time use , 2017, Image Vis. Comput..

[15]  Zhe L. Lin,et al.  Nonparametric Context Modeling of Local Appearance for Pose- and Expression-Robust Facial Landmark Localization , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Junzhou Huang,et al.  Pose-Free Facial Landmark Fitting via Optimized Part Mixtures and Cascaded Deformable Shape Model , 2013, 2013 IEEE International Conference on Computer Vision.