Face Swapping for Solving Collateral Privacy Issues in Multimedia Analytics

A wide range of components of multimedia analytics systems relies on visual content that is used for supervised (e.g., classification) and unsupervised (e.g., clustering) machine learning methods. This content may contain privacy sensitive information, e.g., show faces of persons. In many cases it is just an inevitable side-effect that persons appear in the content, and the application may not require identification – a situation which we call “collateral privacy issues”. We propose de-identification of faces in images by using a generative adversarial network to generate new face images, and use them to replace faces in the original images. We demonstrate that face swapping does not impact the performance of visual descriptor matching and extraction.

[1]  Richa Singh,et al.  SWAPPED! Digital face presentation attack detection via weighted local magnitude pattern , 2017, 2017 IEEE International Joint Conference on Biometrics (IJCB).

[2]  Shree K. Nayar,et al.  Face swapping: automatically replacing faces in photographs , 2008, SIGGRAPH 2008.

[3]  Li Meng,et al.  Efficient approach to de-identifying faces in videos , 2017, IET Signal Process..

[4]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[5]  Jean Le Feuvre,et al.  Real-time selective encryption solution based on ROI for MPEG-A visual identity management AF , 2017, 2017 22nd International Conference on Digital Signal Processing (DSP).

[6]  Alexandros André Chaaraoui,et al.  Visual privacy protection methods: A survey , 2015, Expert Syst. Appl..

[7]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[8]  Jean-Philippe Domenger,et al.  Face de-identification with expressions preservation , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[9]  Ling-yu Duan,et al.  Rate-adaptive Compact Fisher Codes for Mobile Visual Search , 2014, IEEE Signal Processing Letters.

[10]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[11]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[12]  Mohan S. Kankanhalli,et al.  W3-privacy: understanding what, when, and where inference channels in multi-camera surveillance video , 2012, Multimedia Tools and Applications.

[13]  Shigeo Morishima,et al.  RSGAN: face swapping and editing using face and hair representation in latent spaces , 2018, SIGGRAPH Posters.

[14]  Atta Badii,et al.  Overview of the MediaEval 2013 Visual Privacy Task , 2013, MediaEval.

[15]  Jianping Fan,et al.  Intended human object detection for automatically protecting privacy in mobile video surveillance , 2011, Multimedia Systems.

[16]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[17]  Masashi Nishiyama,et al.  Facial deblur inference to improve recognition of blurred fFaces , 2009, CVPR.

[18]  Michael Isard,et al.  Object retrieval with large vocabularies and fast spatial matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Sachit Mahajan,et al.  SwapItUp: A Face Swap Application for Privacy Protection , 2017, 2017 IEEE 31st International Conference on Advanced Information Networking and Applications (AINA).

[20]  Vicki Bruce,et al.  Evaluating the effectiveness of pixelation and blurring on masking the identity of familiar faces , 2001 .

[21]  Timo Ahonen,et al.  Recognition of blurred faces using Local Phase Quantization , 2008, 2008 19th International Conference on Pattern Recognition.

[22]  Ying Zhang,et al.  Automated face swapping and its detection , 2017, 2017 IEEE 2nd International Conference on Signal and Image Processing (ICSIP).

[23]  Wen Gao,et al.  Compact Deep Invariant Descriptors for Video Retrieval , 2017, 2017 Data Compression Conference (DCC).

[24]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[25]  Bradley Malin,et al.  Preserving privacy by de-identifying face images , 2005, IEEE Transactions on Knowledge and Data Engineering.

[26]  Tal Hassner,et al.  On Face Segmentation, Face Swapping, and Face Perception , 2017, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[27]  Masashi Nishiyama,et al.  Recognition of Blurred Faces via Facial Deblurring Combined with Blur-Tolerant Descriptors , 2010, 2010 20th International Conference on Pattern Recognition.

[28]  Davis E. King,et al.  Dlib-ml: A Machine Learning Toolkit , 2009, J. Mach. Learn. Res..