Photorealistic Face De-Identification by Aggregating Donors' Face Components

With the adoption of pervasive surveillance systems and the development of efficient automatic face matchers, the question of preserving privacy becomes paramount. In this context, automated face de-identification is revived. Typical solutions based on eyes masking or pixelization, while commonly used in news broadcasts, produce very unnatural images. More sophisticated solutions were sparingly introduced in the literature, but they fail to account for fundamental constraints such as the visual likeliness of de-identified images. In contrast, we identify essential principles and build upon efficient techniques to derive an automated face de-identification solution meeting our predefined criteria. More specifically, our approach relies on a set of face donors from which it can borrow various face components (eyes, chin, etc.). Faces are then de-identified by substituting their own face components with the donors’ ones, in such a way that an automatic face matcher is fooled while the appearance of the generated faces are as close as possible to original faces. Experiments on several datasets validate the approach and show its ability both in terms of privacy preservation and visual quality.

[1]  Yuan Lin,et al.  Face replacement with large-pose differences , 2012, ACM Multimedia.

[2]  John T. Stasko,et al.  Evaluating image filtering based techniques in media space applications , 1998, CSCW '98.

[3]  Anil K. Jain,et al.  Component-Based Representation in Automated Face Recognition , 2013, IEEE Transactions on Information Forensics and Security.

[4]  Giovanni Vigna,et al.  Getting the Face Behind the Squares: Reconstructing Pixelized Video Streams , 2011, WOOT.

[5]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Bradley Malin,et al.  Preserving privacy by de-identifying face images , 2005, IEEE Transactions on Knowledge and Data Engineering.

[7]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[8]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[9]  James L. Crowley,et al.  Perceptual user interfaces: things that see , 2000, CACM.

[10]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[11]  Luc Van Gool,et al.  Unsupervised face alignment by robust nonrigid mapping , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[12]  Jan Kautz,et al.  Visio-lization: generating novel facial images , 2009, ACM Trans. Graph..

[13]  Li Bai,et al.  Cosine Similarity Metric Learning for Face Verification , 2010, ACCV.

[14]  Simon Baker,et al.  2D vs. 3D Deformable Face Models: Representational Power, Construction, and Real-Time Fitting , 2007, International Journal of Computer Vision.

[15]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[16]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[17]  Adam Schmidt,et al.  The put face database , 2008 .

[18]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Markus Dürmuth,et al.  Achieving Anonymity against Major Face Recognition Algorithms , 2013, Communications and Multimedia Security.

[20]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  P. J. Narayanan,et al.  Person De-Identification in Videos , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Scott E. Hudson,et al.  Techniques for addressing fundamental privacy and disruption tradeoffs in awareness support systems , 1996, CSCW '96.

[23]  Tal Hassner,et al.  Viewing Real-World Faces in 3D , 2013, 2013 IEEE International Conference on Computer Vision.

[24]  Touradj Ebrahimi,et al.  A framework for the validation of privacy protection solutions in video surveillance , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[25]  Ralph Gross,et al.  Semi-supervised learning of multi-factor models for face de-identification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Vicki Bruce,et al.  Evaluating the effectiveness of pixelation and blurring on masking the identity of familiar faces , 2001 .

[27]  Shree K. Nayar,et al.  Face swapping: automatically replacing faces in photographs , 2008, SIGGRAPH 2008.

[28]  Stephen Milborrow The MUCT Landmarked Face Database , 2010 .

[29]  Zhe L. Lin,et al.  Exemplar-Based Face Parsing , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Christopher Edwards,et al.  The effects of filtered video on awareness and privacy , 2000, CSCW '00.

[31]  Chih-Yuan Yang,et al.  Structured Face Hallucination , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.