k-Same-Net: k-Anonymity with Generative Deep Neural Networks for Face Deidentification †

Image and video data are today being shared between government entities and other relevant stakeholders on a regular basis and require careful handling of the personal information contained therein. A popular approach to ensure privacy protection in such data is the use of deidentification techniques, which aim at concealing the identity of individuals in the imagery while still preserving certain aspects of the data after deidentification. In this work, we propose a novel approach towards face deidentification, called k-Same-Net, which combines recent Generative Neural Networks (GNNs) with the well-known k-Anonymitymechanism and provides formal guarantees regarding privacy protection on a closed set of identities. Our GNN is able to generate synthetic surrogate face images for deidentification by seamlessly combining features of identities used to train the GNN model. Furthermore, it allows us to control the image-generation process with a small set of appearance-related parameters that can be used to alter specific aspects (e.g., facial expressions, age, gender) of the synthesized surrogate images. We demonstrate the feasibility of k-Same-Net in comprehensive experiments on the XM2VTS and CK+ datasets. We evaluate the efficacy of the proposed approach through reidentification experiments with recent recognition models and compare our results with competing deidentification techniques from the literature. We also present facial expression recognition experiments to demonstrate the utility-preservation capabilities of k-Same-Net. Our experimental results suggest that k-Same-Net is a viable option for facial deidentification that exhibits several desirable characteristics when compared to existing solutions in this area.

[1]  Bradley Malin,et al.  Preserving privacy by de-identifying face images , 2005, IEEE Transactions on Knowledge and Data Engineering.

[2]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[3]  Isabel Martinez-Ponte,et al.  ROBUST HUMAN FACE HIDING ENSURING PRIVACY , 2004 .

[4]  Peter Peer,et al.  Ear recognition: More than a survey , 2016, Neurocomputing.

[5]  Yoshua Bengio,et al.  Generative Adversarial Networks , 2014, ArXiv.

[6]  Slobodan Ribaric,et al.  An approach to the de-identification of faces in different poses , 2014, 2014 37th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[7]  Junmo Kim,et al.  Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  Albert Ali Salah,et al.  Facial Expression Recognition in the Wild Using Improved Dense Trajectories and Fisher Vector Encoding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[9]  Li Meng,et al.  Distinguishable de-identified faces , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[10]  Philip S. Yu,et al.  A General Survey of Privacy-Preserving Data Mining Models and Algorithms , 2008, Privacy-Preserving Data Mining.

[11]  Luc Vandendorpe,et al.  Face Verification Competition on the XM2VTS Database , 2003, AVBPA.

[12]  Frédéric Jurie,et al.  Puzzling face verification algorithms for privacy protection , 2014, 2014 IEEE International Workshop on Information Forensics and Security (WIFS).

[13]  Rima Kilany,et al.  Towards Flexible K-Anonymity , 2016, KESW.

[14]  Alice Caplier,et al.  Face recognition using the POEM descriptor , 2012, Pattern Recognit..

[15]  Patrick Pérez,et al.  Automatic Face Reenactment , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Edoardo M. Airoldi,et al.  Integrating Utility into Face De-identification , 2005, Privacy Enhancing Technologies.

[17]  Nikola Pavesic,et al.  De-identification for privacy protection in multimedia content: A survey , 2016, Signal Process. Image Commun..

[18]  Jaakko Lehtinen,et al.  Progressive Growing of GANs for Improved Quality, Stability, and Variation , 2017, ICLR.

[19]  Xu Yang,et al.  Deep Age Distribution Learning for Apparent Age Estimation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[20]  Nello Cristianini,et al.  Learning to classify gender from four million images , 2015, Pattern Recognit. Lett..

[21]  Yan Li,et al.  A Study on Apparent Age Estimation , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[22]  Yu Yang,et al.  PIEFA: Personalized Incremental and Ensemble Face Alignment , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[23]  Alina Campan,et al.  P-Sensitive K-Anonymity with Generalization Constraints , 2010, Trans. Data Priv..

[24]  Tal Hassner,et al.  Age and gender classification using convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  Hrvoje Gebavi,et al.  and Communication Technology, Electronics and Microelectronics (MIPRO) , 2015 .

[26]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[27]  Erik Blasch,et al.  GARP-face: Balancing privacy protection and utility preservation in face de-identification , 2014, IEEE International Joint Conference on Biometrics.

[28]  Touradj Ebrahimi,et al.  Subjective study of privacy filters in video surveillance , 2012, 2012 IEEE 14th International Workshop on Multimedia Signal Processing (MMSP).

[29]  Xiaoming Liu,et al.  Large-Pose Face Alignment via CNN-Based Dense 3D Model Fitting , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Séverine Dubuisson,et al.  Pairwise Conditional Random Forests for Facial Expression Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[31]  Xiaoming Liu,et al.  Representation Learning by Rotating Your Faces , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Mohammad H. Mahoor,et al.  Facial Expression Recognition from World Wild Web , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[33]  Amit K. Roy-Chowdhury,et al.  Re-Identification in the Function Space of Feature Warps , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Aladdin M. Ariyaeeinia,et al.  Retaining expressions on de-identified faces , 2014, 2014 37th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[35]  Zhaoqiang Xia,et al.  Towards Facial Expression Recognition in the Wild: A New Database and Deep Recognition System , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[36]  Ivan Sikiric,et al.  Face, hairstyle and clothing colour de-identification in video sequences , 2017, IET Signal Process..

[37]  Touradj Ebrahimi,et al.  A framework for the validation of privacy protection solutions in video surveillance , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[38]  ASHWIN MACHANAVAJJHALA,et al.  L-diversity: privacy beyond k-anonymity , 2006, 22nd International Conference on Data Engineering (ICDE'06).

[39]  Justus Thies,et al.  Face2Face: real-time face capture and reenactment of RGB videos , 2019, Commun. ACM.

[40]  Rama Chellappa,et al.  HyperFace: A Deep Multi-Task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Luis Enrique Sucar,et al.  Foveation: an alternative method to simultaneously preserve privacy and information in face images , 2017, J. Electronic Imaging.

[42]  Ralph Gross,et al.  Model-Based Face De-Identification , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[43]  Charu C. Aggarwal,et al.  On the design and quantification of privacy preserving data mining algorithms , 2001, PODS.

[44]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[45]  Ninghui Li,et al.  t-Closeness: Privacy Beyond k-Anonymity and l-Diversity , 2007, 2007 IEEE 23rd International Conference on Data Engineering.

[46]  Xiaoou Tang,et al.  Facial Landmark Detection by Deep Multi-task Learning , 2014, ECCV.

[47]  Blaz Meden,et al.  κ-Same-Net: Neural-Network-Based Face Deidentification , 2017, 2017 International Conference and Workshop on Bioinspired Intelligence (IWOBI).

[48]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[49]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[50]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[51]  Marcus Liwicki,et al.  DeXpression: Deep Convolutional Neural Network for Expression Recognition , 2015, ArXiv.

[52]  Klemen Grm,et al.  Strengths and weaknesses of deep learning models for face recognition against image degradations , 2017, IET Biom..

[53]  Ralph Gross,et al.  Semi-supervised learning of multi-factor models for face de-identification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[55]  Thomas Brox,et al.  Learning to Generate Chairs, Tables and Cars with Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Roberto Paredes,et al.  Local Deep Neural Networks for gender recognition , 2016, Pattern Recognit. Lett..

[57]  Frédéric Jurie,et al.  Photorealistic Face De-Identification by Aggregating Donors' Face Components , 2014, ACCV.

[58]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[59]  H. T. Mouftah,et al.  Chaos-cryptography based privacy preservation technique for video surveillance , 2011, Multimedia Systems.

[60]  Peter Peer,et al.  Frame–based classification for cross-speed gait recognition , 2017, Multimedia Tools and Applications.

[61]  Rima Kilany,et al.  Towards internal privacy and flexible K-anonymity , 2015, 2015 International Conference on Applied Research in Computer Science and Engineering (ICAR).

[62]  T. Truta,et al.  Constrained k-Anonymity : Privacy with Generalization Boundaries , 2022 .

[63]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[64]  Rama Chellappa,et al.  Unconstrained Age Estimation with Deep Convolutional Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[65]  Markus Dürmuth,et al.  Achieving Anonymity against Major Face Recognition Algorithms , 2013, Communications and Multimedia Security.

[66]  Skyler T. Hawk,et al.  Presentation and validation of the Radboud Faces Database , 2010 .

[67]  Ivan Sikiric,et al.  I Know That Person: Generative Full Body and Face De-identification of People in Images , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[68]  Jian Sun,et al.  Face Alignment at 3000 FPS via Regressing Local Binary Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[69]  Gang Hua,et al.  Ordinal Regression with Multiple Output CNN for Age Estimation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[70]  P. J. Narayanan,et al.  Person De-Identification in Videos , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[71]  Peter Peer,et al.  Human Skeleton Model Based Dynamic Features for Walking Speed Invariant Gait Recognition , 2014 .

[72]  Raja Bala,et al.  Semi-supervised Conditional GANs , 2017, ArXiv.

[73]  Blaz Meden,et al.  Face Deidentification with Generative Deep Neural Networks , 2017, IET Signal Process..

[74]  Xiangyu Zhu,et al.  Face Alignment in Full Pose Range: A 3D Total Solution , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[75]  Touradj Ebrahimi,et al.  Using warping for privacy protection in video surveillance , 2013, 2013 18th International Conference on Digital Signal Processing (DSP).

[76]  Minghua Chen,et al.  Hiding privacy information in video surveillance system , 2005, IEEE International Conference on Image Processing 2005.

[77]  Latanya Sweeney,et al.  k-Anonymity: A Model for Protecting Privacy , 2002, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[78]  Lin Wu,et al.  Deep adaptive feature embedding with local sample distributions for person re-identification , 2017, Pattern Recognit..

[79]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[80]  Sebastian Nowozin,et al.  f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization , 2016, NIPS.

[81]  B. K. Julsing,et al.  Face Recognition with Local Binary Patterns , 2012 .

[82]  Aruna Shenoy,et al.  Retaining Expression on De-identified Faces , 2017, SPECOM.

[83]  Jean-Philippe Domenger,et al.  Face de-identification with expressions preservation , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[84]  Rok Gajsek,et al.  Emotion recognition using linear transformations in combination with video , 2009, INTERSPEECH.

[85]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[86]  Reuben A. Farrugia Reversible De-Identification for lossless image compression using Reversible Watermarking , 2014, 2014 37th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).