Encryption-Free Framework of Privacy-Preserving Image Recognition for Photo-Based Information Services

Nowadays, mobile devices, such as smartphones, have been widely used all over the world. In addition, the performance of image recognition has drastically increased with deep learning technologies. From these backgrounds, some photo-based information services provided in a client–server architecture are getting popular: client users take a photo of a certain spot and send it to a server, while the server identifies the spot with an image recognizer and returns its related information to the users. However, this kind of client–server image recognition can cause a privacy issue because image recognition results are sometimes privacy-sensitive. To tackle the privacy issue, in this paper, we propose a framework of privacy-preserving image recognition called EnfPire, in which the server cannot uniquely determine the recognition result but client users can do so. An overview of EnfPire is as follows. First, client users extract a visual feature from their taken photo and transform it so that the server cannot uniquely determine the recognition result. Then, the users send the transformed feature to the server that returns a set of candidates of the recognition result to the users. Finally, the users compare the candidates to the original visual feature for obtaining the final result. Our experimental results demonstrate that EnfPire successfully degrades the server’s spot-recognition accuracy from 99.8% to 41.4% while keeping 86.9% of the spot-recognition accuracy on the user side.

[1]  Yuan Yan Tang,et al.  An image classification method that considers privacy-preservation , 2016, Neurocomputing.

[2]  Shuicheng Yan,et al.  Hybrid CNN and Dictionary-Based Models for Scene Recognition and Domain Adaptation , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Takahiro Hara,et al.  Dummy-Based User Location Anonymization Under Real-World Constraints , 2016, IEEE Access.

[4]  Touradj Ebrahimi,et al.  Semantic video analysis for adaptive content delivery and automatic description , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Yunhao Liu,et al.  PIC: Enable Large-Scale Privacy Preserving Content-Based Image Search on Cloud , 2015, IEEE Transactions on Parallel and Distributed Systems.

[6]  Daewon Kim,et al.  Non-Marker based Mobile Augmented Reality and its Applications using Object Recognition , 2012, J. Univers. Comput. Sci..

[7]  Shree K. Nayar,et al.  Face swapping: automatically replacing faces in photographs , 2008, SIGGRAPH 2008.

[8]  Xiaolin Hu,et al.  Recurrent convolutional neural network for object recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Masayuki Mukunoki,et al.  Privacy-Protected Camera for the Sensing Web , 2010, IPMU.

[10]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[11]  Jong Kim,et al.  Protecting location privacy using location semantics , 2011, KDD.

[12]  Tetsuji Satoh,et al.  An anonymous communication technique using dummies for location-based services , 2005, ICPS '05. Proceedings. International Conference on Pervasive Services, 2005..

[13]  Laurent Amsaleg,et al.  A Privacy-Preserving Framework for Large-Scale Content-Based Information Retrieval , 2015, IEEE Transactions on Information Forensics and Security.

[14]  Young-Geun Kim,et al.  Implementation of Augmented Reality System for Smartphone Advertisements , 2014, MUE 2014.

[15]  Luis Herranz,et al.  Scene Recognition with CNNs: Objects, Scales and Dataset Bias , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Lars Kulik,et al.  A Formal Model of Obfuscation and Negotiation for Location Privacy , 2005, Pervasive.

[17]  Yi Zhu,et al.  Towards Privacy-Preserving Content-Based Image Retrieval in Cloud Computing , 2018, IEEE Transactions on Cloud Computing.

[18]  Chris Clifton,et al.  Privately Computing a Distributed k-nn Classifier , 2004, PKDD.

[19]  Volkmar Frinken,et al.  Selective Concealment of Characters for Privacy Protection , 2014, 2014 22nd International Conference on Pattern Recognition.

[20]  Kannan Ramchandran,et al.  Toward efficient, privacy-aware media classification on public databases , 2014, ICMR.

[21]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[22]  Mikhail J. Atallah,et al.  Secure and Private Outsourcing of Shape-Based Feature Extraction , 2013, ICICS.

[23]  Jorma Laaksonen,et al.  Convolutional Network Features for Scene Recognition , 2014, ACM Multimedia.

[24]  Cong Wang,et al.  Towards Efficient Privacy-preserving Image Feature Extraction in Cloud Computing , 2014, ACM Multimedia.

[25]  Dong Yu,et al.  Deep Learning: Methods and Applications , 2014, Found. Trends Signal Process..

[26]  Noboru Babaguchi,et al.  A Framework of Privacy-Preserving Image Recognition for Image-Based Information Services , 2017, MMM.

[27]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[28]  Stefan Katzenbeisser,et al.  Privacy-Preserving Face Recognition , 2009, Privacy Enhancing Technologies.

[29]  Serge J. Belongie,et al.  Removing pedestrians from Google street view images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[30]  Christopher Edwards,et al.  The effects of filtered video on awareness and privacy , 2000, CSCW '00.

[31]  Noboru Babaguchi,et al.  Facial expression preserving privacy protection using image melding , 2015, 2015 IEEE International Conference on Multimedia and Expo (ICME).

[32]  Bolei Zhou,et al.  Learning Deep Features for Scene Recognition using Places Database , 2014, NIPS.

[33]  Soo-Chang Pei,et al.  Image Feature Extraction in Encrypted Domain With Privacy-Preserving SIFT , 2012, IEEE Transactions on Image Processing.

[34]  Marco Zennaro,et al.  Large-scale privacy protection in Google Street View , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[35]  Min Wu,et al.  Enabling search over encrypted multimedia databases , 2009, Electronic Imaging.

[36]  Carman Neustaedter,et al.  Blur filtration fails to preserve privacy for home-based video conferencing , 2006, TCHI.

[37]  Yi-Chong Zeng,et al.  Scene Feature Recognition-Enabled Framework for Mobile Service Information Query System , 2015, HCI.

[38]  Ahmad-Reza Sadeghi,et al.  Efficient Privacy-Preserving Face Recognition , 2009, ICISC.

[39]  Edoardo M. Airoldi,et al.  Integrating Utility into Face De-identification , 2005, Privacy Enhancing Technologies.

[40]  Jitendra Malik,et al.  Analyzing the Performance of Multilayer Neural Networks for Object Recognition , 2014, ECCV.

[41]  Chunming Tang,et al.  Privacy-preserving face recognition with outsourced computation , 2016, Soft Comput..

[42]  Julien Bringer,et al.  Privacy-Preserving Biometric Identification Using Secure Multiparty Computation: An Overview and Recent Trends , 2013, IEEE Signal Processing Magazine.

[43]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[44]  Muttukrishnan Rajarajan,et al.  Privacy-Preserving Multi-Class Support Vector Machine for Outsourcing the Data Classification in Cloud , 2014, IEEE Transactions on Dependable and Secure Computing.

[45]  Ernesto Damiani,et al.  Location Privacy Protection Through Obfuscation-Based Techniques , 2007, DBSec.

[46]  Ralph Gross,et al.  Model-Based Face De-Identification , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[47]  Wei-Ta Chu,et al.  A Privacy-Preserving Bipartite Graph Matching Framework for Multimedia Analysis and Retrieval , 2015, ICMR.

[48]  João Leitão,et al.  Privacy-Preserving Content-Based Image Retrieval in the Cloud , 2014, 2015 IEEE 34th Symposium on Reliable Distributed Systems (SRDS).

[49]  Noboru Babaguchi,et al.  PriSurv: Privacy Protected Video Surveillance System Using Adaptive Visual Abstraction , 2008, MMM.

[50]  Yu Zhang,et al.  Preserving User Location Privacy in Mobile Data Management Infrastructures , 2006, Privacy Enhancing Technologies.

[51]  Rin-ichiro Taniguchi,et al.  Anonymous Camera for Privacy Protection , 2014, 2014 22nd International Conference on Pattern Recognition.

[52]  Soo-Chang Pei,et al.  Secure and robust SIFT , 2009, ACM Multimedia.

[53]  Cong Wang,et al.  SecSIFT: Secure Image SIFT Feature Extraction in Cloud Computing , 2016, ACM Trans. Multim. Comput. Commun. Appl..