Hide and seek: Uncovering facial occlusion with variable-threshold robust PCA

Face images are very important in human social activities, which can be severely hampered when they are corrupted by occluders such as eyeglasses, face marks, and scarfs. Existing methods for removing occlusions in face images can be grouped into three broad categories, namely PCA, robust PCA (RPCA), and sparse coding. The major weaknesses of these methods are inconsistent performance across test conditions and possible corruption of unoccluded part of the recovered target image. This paper presents variable-threshold RPCA (VRPCA) method based on RPCA with variable thresholding. Comprehensive tests show that VRPCA is able to preserve the unoccluded parts of the target image with practically zero error. Compared to existing methods, it is more accurate, reliable, and consistent across various test conditions.

[1]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[2]  Seong-Whan Lee,et al.  Reconstruction of Partially Damaged Face Images Based on a Morphable Face Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Yasuyuki Saito,et al.  Estimation of eyeglassless facial images using principal component analysis , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[4]  Dao-Qing Dai,et al.  Structured Sparse Error Coding for Face Recognition With Occlusion , 2013, IEEE Transactions on Image Processing.

[5]  Nojun Kwak,et al.  Detection and Recovery of Occluded Face Images based on Correlation between Pixels , 2012, ICPRAM.

[6]  Horst Bischof,et al.  Robust Recognition Using Eigenimages , 2000, Comput. Vis. Image Underst..

[7]  Yu-Chiang Frank Wang,et al.  Low-rank matrix recovery with structural incoherence for robust face recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Sang Chul Ahn,et al.  Glasses removal from facial image using recursive error compensation , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[10]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[11]  LeeSeong-Whan,et al.  Reconstruction of Partially Damaged Face Images Based on a Morphable Face Model , 2003 .

[12]  J. Tao,et al.  Reconstruction of Partially Occluded Face by Fast Recursive PCA , 2007, 2007 International Conference on Computational Intelligence and Security Workshops (CISW 2007).

[13]  Dahua Lin,et al.  Quality-Driven Face Occlusion Detection and Recovery , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Harry Shum,et al.  Automatic eyeglasses removal from face images , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Xiaopeng Zhang,et al.  Illumination compensation via low rank matrix completion for multiview video coding , 2013, 2013 IEEE International Conference on Image Processing.

[16]  Xudong Jiang,et al.  Sparse and Dense Hybrid Representation via Dictionary Decomposition for Face Recognition , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Jing Liu,et al.  Learning Robust Face Representation With Classwise Block-Diagonal Structure , 2014, IEEE Transactions on Information Forensics and Security.

[18]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[19]  Alexei A. Efros,et al.  Image‐based Shaving , 2008, Comput. Graph. Forum.

[20]  Terence Sim,et al.  Using targeted statistics for face regeneration , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[21]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[22]  Yuan Cheng,et al.  Background Recovery by Fixed-Rank Robust Principal Component Analysis , 2013, CAIP.

[23]  John P. Lewis,et al.  Face Inpainting with Local Linear Representations , 2004, BMVC.

[24]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[25]  Feng Liu,et al.  Depth Enhancement via Low-Rank Matrix Completion , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Alexandre Bernardino,et al.  Matrix Completion for Weakly-Supervised Multi-Label Image Classification , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Baoxin Li,et al.  Joint Sparsity Model with Matrix Completion for an ensemble of face images , 2010, 2010 IEEE International Conference on Image Processing.

[28]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[29]  Jian Yang,et al.  Robust Low-Rank Regularized Regression for Face Recognition with Occlusion , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[30]  Horst Bischof,et al.  Fast-Robust PCA , 2009, SCIA.

[31]  Chunheng Wang,et al.  Sparse representation for face recognition based on discriminative low-rank dictionary learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Thierry Bouwmans,et al.  Robust PCA via Principal Component Pursuit: A review for a comparative evaluation in video surveillance , 2014, Comput. Vis. Image Underst..

[33]  Jean-Luc Dugelay,et al.  Inpainting of sparse occlusion in face recognition , 2012, 2012 19th IEEE International Conference on Image Processing.

[34]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Koichi Ito,et al.  Restoring occluded regions using FW-PCA for face recognition , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[36]  Jian Yang,et al.  Robust sparse coding for face recognition , 2011, CVPR 2011.

[37]  Yun Fu,et al.  Discriminative dictionary learning with low-rank regularization for face recognition , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).