DP-Image: Differential Privacy for Image Data in Feature Space

The excessive use of images in social networks, government databases, and industrial applications has posed great privacy risks and raised serious concerns from the public. Even though differential privacy (DP) is a widely accepted criterion that can provide a provable privacy guarantee, the application of DP on unstructured data such as images is not trivial due to the lack of a clear qualification on the meaningful difference between any two images. In this paper, for the first time, we introduce a novel notion of image aware differential privacy, referred to as DP-image, that can protect user’s personal information in images, from both human and AI adversaries. The DP-Image definition is formulated as an extended version of traditional differential privacy, considering the distance measurements between feature space vectors of images. Then we propose a mechanism to achieve DP-Image by adding noise to an image feature vector. Finally, we conduct experiments with a case study on face image privacy. Our results show that the proposed DP-Image method provides excellent DP protection on images, with a controllable distortion to faces.

[1]  W. Feng,et al.  On the (Im)Practicality of Adversarial Perturbation for Image Privacy , 2020, Proc. Priv. Enhancing Technol..

[2]  Tao Li,et al.  AnonymousNet: Natural Face De-Identification With Measurable Privacy , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[4]  Bo Liu,et al.  A Hybrid Model for Natural Face De-Identiation with Adjustable Privacy , 2020, 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP).

[5]  Jian Xiong,et al.  Protecting Multimedia Privacy from Both Humans and AI , 2019, 2019 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB).

[6]  Sebastian Nowozin,et al.  Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks , 2017, ICML.

[7]  Luc Van Gool,et al.  Natural and Effective Obfuscation by Head Inpainting , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  João Paulo Pesce,et al.  Privacy attacks in social media using photo tagging networks: a case study with Facebook , 2012, PSOSM '12.

[9]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[10]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Timo Aila,et al.  A Style-Based Generator Architecture for Generative Adversarial Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Seong Joon Oh,et al.  Faceless Person Recognition: Privacy Implications in Social Media , 2016, ECCV.

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Suman Jana,et al.  Certified Robustness to Adversarial Examples with Differential Privacy , 2018, 2019 IEEE Symposium on Security and Privacy (SP).

[15]  Heng Xu,et al.  CoPE: Enabling collaborative privacy management in online social networks , 2011, J. Assoc. Inf. Sci. Technol..

[16]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Xiaogang Wang,et al.  Visual Tracking with Fully Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  Liyue Fan,et al.  Image Pixelization with Differential Privacy , 2018, DBSec.

[19]  Bo Liu,et al.  When Machine Learning Meets Privacy , 2020, ACM Comput. Surv..

[20]  Liyue Fan,et al.  Practical Image Obfuscation with Provable Privacy , 2019, 2019 IEEE International Conference on Multimedia and Expo (ICME).

[21]  Stefanos Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Jaakko Lehtinen,et al.  Analyzing and Improving the Image Quality of StyleGAN , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Sepp Hochreiter,et al.  GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium , 2017, NIPS.

[24]  Ting Wang,et al.  DEEPSEC: A Uniform Platform for Security Analysis of Deep Learning Model , 2019, 2019 IEEE Symposium on Security and Privacy (SP).

[25]  Weiming Zhang,et al.  Protecting Privacy in Shared Photos via Adversarial Examples Based Stealth , 2017, Secur. Commun. Networks.

[26]  Daniel Cohen-Or,et al.  Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[28]  Anna Cinzia Squicciarini,et al.  Towards automatic privacy management in Web 2.0 with semantic analysis on annotations , 2009, 2009 5th International Conference on Collaborative Computing: Networking, Applications and Worksharing.

[29]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[30]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[31]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[32]  Ben Y. Zhao,et al.  Fawkes: Protecting Privacy against Unauthorized Deep Learning Models , 2020, USENIX Security Symposium.

[33]  Ling Shao,et al.  Pyramidal Convolution: Rethinking Convolutional Neural Networks for Visual Recognition , 2020, ArXiv.

[34]  Martin J. Wainwright,et al.  Local privacy and statistical minimax rates , 2013, 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[35]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Subhransu Maji,et al.  Deep filter banks for texture recognition and segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Cynthia Dwork,et al.  Differential Privacy: A Survey of Results , 2008, TAMC.

[38]  Vitaly Shmatikov,et al.  Defeating Image Obfuscation with Deep Learning , 2016, ArXiv.

[39]  Fang Wen,et al.  FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping , 2019, ArXiv.

[40]  Bernt Schiele,et al.  A Hybrid Model for Identity Obfuscation by Face Replacement , 2018, ECCV.

[41]  Tianqing Zhu,et al.  Adversaries or allies? Privacy and deep learning in big data era , 2019, Concurr. Comput. Pract. Exp..

[42]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[43]  Fan Yang,et al.  Privacy-Protective-GAN for Privacy Preserving Face De-Identification , 2019, Journal of Computer Science and Technology.

[44]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[45]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[46]  Tribhuvanesh Orekondy,et al.  InfoScrub: Towards Attribute Privacy by Targeted Obfuscation , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[47]  Seong Joon Oh,et al.  Adversarial Image Perturbation for Privacy Protection A Game Theory Perspective , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[48]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[49]  Edward S. Dove,et al.  The EU General Data Protection Regulation: Implications for International Scientific Research in the Digital Era , 2018 .