GAN-Based Differential Private Image Privacy Protection Framework for the Internet of Multimedia Things

With the development of the Internet of Multimedia Things (IoMT), an increasing amount of image data is collected by various multimedia devices, such as smartphones, cameras, and drones. This massive number of images are widely used in each field of IoMT, which presents substantial challenges for privacy preservation. In this paper, we propose a new image privacy protection framework in an effort to protect the sensitive personal information contained in images collected by IoMT devices. We aim to use deep neural network techniques to identify the privacy-sensitive content in images, and then protect it with the synthetic content generated by generative adversarial networks (GANs) with differential privacy (DP). Our experiment results show that the proposed framework can effectively protect users’ privacy while maintaining image utility.

[1]  Tae Hyun Kim,et al.  Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Yuan-Yuan Liu,et al.  DeblurGAN+: Revisiting blind motion deblurring using conditional adversarial networks , 2020, Signal Process..

[3]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  Bo Liu,et al.  A Hybrid Model for Natural Face De-Identiation with Adjustable Privacy , 2020, 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP).

[5]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[6]  Jian Xiong,et al.  Protecting Multimedia Privacy from Both Humans and AI , 2019, 2019 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB).

[7]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[8]  Latanya Sweeney,et al.  k-Anonymity: A Model for Protecting Privacy , 2002, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[9]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.

[10]  Cynthia Rudin,et al.  PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Vitaly Shmatikov,et al.  Defeating Image Obfuscation with Deep Learning , 2016, ArXiv.

[12]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[13]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[14]  Tianqing Zhu,et al.  Security and privacy in 6G networks: New areas and new challenges , 2020, Digit. Commun. Networks.

[15]  Jens Grossklags,et al.  Third-party apps on Facebook: privacy and the illusion of control , 2011, CHIMIT '11.

[16]  Paul C. van Oorschot,et al.  Privacy-enhanced sharing of personal content on the web , 2008, WWW.

[17]  Cynthia Dwork,et al.  Differential Privacy , 2006, ICALP.

[18]  Jiri Matas,et al.  DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Anna Cinzia Squicciarini,et al.  Towards automatic privacy management in Web 2.0 with semantic analysis on annotations , 2009, 2009 5th International Conference on Collaborative Computing: Networking, Applications and Worksharing.

[20]  Flora Graham,et al.  Daily briefing: San Francisco bans facial-recognition technology , 2019, Nature.

[21]  Xiangliang Zhang,et al.  CreditCoin: A Privacy-Preserving Blockchain-Based Incentive Announcement Network for Communications of Smart Vehicles , 2018, IEEE Transactions on Intelligent Transportation Systems.

[22]  João Paulo Pesce,et al.  Privacy attacks in social media using photo tagging networks: a case study with Facebook , 2012, PSOSM '12.

[23]  Tianqing Zhu,et al.  Adversaries or allies? Privacy and deep learning in big data era , 2019, Concurr. Comput. Pract. Exp..

[24]  Xuefeng Liu,et al.  Privacy-Preserving Reputation Management for Edge Computing Enhanced Mobile Crowdsensing , 2019, IEEE Transactions on Services Computing.

[25]  ASHWIN MACHANAVAJJHALA,et al.  L-diversity: privacy beyond k-anonymity , 2006, 22nd International Conference on Data Engineering (ICDE'06).

[26]  Timo Aila,et al.  A Style-Based Generator Architecture for Generative Adversarial Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Peter Wonka,et al.  Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space? , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[28]  Petros Maragos,et al.  An inpainting system for automatic image structure - texture restoration with text removal , 2008, 2008 15th IEEE International Conference on Image Processing.

[29]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Sepp Hochreiter,et al.  Self-Normalizing Neural Networks , 2017, NIPS.

[31]  Bo Liu,et al.  When Machine Learning Meets Privacy , 2020, ACM Comput. Surv..

[32]  Sheng Wen,et al.  Faces are Protected as Privacy: An Automatic Tagging Framework Against Unpermitted Photo Sharing in Social Media , 2019, IEEE Access.

[33]  Dariu Gavrila,et al.  Privacy Protection in Street-View Panoramas Using Depth and Multi-View Imagery , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[35]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Justus Thies,et al.  Deferred Neural Rendering: Image Synthesis using Neural Textures , 2019 .

[37]  Ninghui Li,et al.  t-Closeness: Privacy Beyond k-Anonymity and l-Diversity , 2007, 2007 IEEE 23rd International Conference on Data Engineering.

[38]  Yu Wang,et al.  A fog-based privacy-preserving approach for distributed signature-based intrusion detection , 2018, J. Parallel Distributed Comput..

[39]  Heng Xu,et al.  CoPE: Enabling collaborative privacy management in online social networks , 2011, J. Assoc. Inf. Sci. Technol..

[40]  Jitendra Malik,et al.  Hypercolumns for object segmentation and fine-grained localization , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Andreas Rössler,et al.  FaceForensics++: Learning to Detect Manipulated Facial Images , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[42]  Wei Xiong,et al.  Foreground-Aware Image Inpainting , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Weiming Zhang,et al.  Protecting Privacy in Shared Photos via Adversarial Examples Based Stealth , 2017, Secur. Commun. Networks.

[44]  Tianqing Zhu,et al.  Hiding Private Information in Images From AI , 2020, ICC 2020 - 2020 IEEE International Conference on Communications (ICC).

[45]  Lucas Theis,et al.  Fast Face-Swap Using Convolutional Neural Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[46]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Saeid Nahavandi,et al.  Deep Learning for Deepfakes Creation and Detection , 2019, ArXiv.

[48]  Cornelia Caragea,et al.  Image Privacy Prediction Using Deep Neural Networks , 2019, ACM Trans. Web.

[49]  Edward S. Dove,et al.  The EU General Data Protection Regulation: Implications for International Scientific Research in the Digital Era , 2018 .

[50]  Jianping Fan,et al.  iPrivacy: Image Privacy Protection by Identifying Sensitive Objects via Deep Multi-Task Learning , 2017, IEEE Transactions on Information Forensics and Security.

[51]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[52]  Yu Wang,et al.  Detecting insider attacks in medical cyber-physical networks based on behavioral profiling , 2020, Future Gener. Comput. Syst..