HERS: Homomorphically Encrypted Representation Search

We present a method to search for a probe (or query) image representation against a large gallery in the encrypted domain. We require that the probe and gallery images be represented in terms of a fixed-length representation, which is typical for representations obtained from learned networks. Our encryption scheme is agnostic to how the fixed-length representation is obtained and can, therefore, be applied to any fixed-length representation in any application domain. Our method, dubbed HERS (Homomorphically Encrypted Representation Search), operates by (i) compressing the representation towards its estimated intrinsic dimensionality, (ii) encrypting the compressed representation using the proposed fully homomorphic encryption scheme, and (iii) searching against a gallery of encrypted representations directly in the encrypted domain, without decrypting them, and with minimal loss of accuracy. Numerical results on large galleries of face, fingerprint, and object datasets such as ImageNet show that, for the first time, accurate and fast image search within the encrypted domain is feasible at scale (296 seconds; 46x speedup over state-of-the-art for face search against a background of 1 million).

[1]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[2]  Ross B. Girshick,et al.  Reducing Overfitting in Deep Networks by Decorrelating Representations , 2015, ICLR.

[3]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[4]  Kang Ryoung Park,et al.  A New Method for Generating an Invariant Iris Private Key Based on the Fuzzy Vault System , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[5]  Sing Bing Kang,et al.  Revealing Scenes by Inverting Structure From Motion Reconstructions , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Fernando Pérez-González,et al.  Fully Private Noninteractive Face Verification , 2013, IEEE Transactions on Information Forensics and Security.

[7]  Anantha Chandrakasan,et al.  Gazelle: A Low Latency Framework for Secure Neural Network Inference , 2018, IACR Cryptol. ePrint Arch..

[8]  Moni Naor,et al.  Visual Cryptography , 1994, Encyclopedia of Multimedia.

[9]  Sharath Pankanti,et al.  Fuzzy Vault for Fingerprints , 2005, AVBPA.

[10]  Craig Gentry,et al.  Implementing Gentry's Fully-Homomorphic Encryption Scheme , 2011, EUROCRYPT.

[11]  Frederik Vercauteren,et al.  Somewhat Practical Fully Homomorphic Encryption , 2012, IACR Cryptol. ePrint Arch..

[12]  Ira Kemelmacher-Shlizerman,et al.  The MegaFace Benchmark: 1 Million Faces for Recognition at Scale , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Anil K. Jain,et al.  Hardening Fingerprint Fuzzy Vault Using Password , 2007, ICB.

[14]  Yong Jae Lee,et al.  Learning to Anonymize Faces for Privacy Preserving Action Detection , 2018, ECCV.

[15]  Patrick Pérez,et al.  Reconstructing an image from its local descriptors , 2011, CVPR 2011.

[16]  Michael Naehrig,et al.  CryptoNets: applying neural networks to encrypted data with high throughput and accuracy , 2016, ICML 2016.

[17]  Oded Goldreich,et al.  The Foundations of Cryptography - Volume 2: Basic Applications , 2001 .

[18]  Ann Cavoukian,et al.  Biometric Encryption , 2011, Encyclopedia of Cryptography and Security.

[19]  Antonio Torralba,et al.  HOGgles: Visualizing Object Detection Features , 2013, 2013 IEEE International Conference on Computer Vision.

[20]  Tatsuya Harada,et al.  Image Reconstruction from Bag-of-Visual-Words , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Vishnu Naresh Boddeti,et al.  On the Intrinsic Dimensionality of Image Representations , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Jung Hee Cheon,et al.  Ghostshell: Secure Biometric Authentication using Integrity-based Homomorphic Evaluations , 2016, IACR Cryptol. ePrint Arch..

[23]  B. V. K. Vijaya Kumar,et al.  A Framework for Binding and Retrieving Class-Specific Information to and from Image Patterns Using Correlation Filters , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Oded Goldreich,et al.  Foundations of Cryptography: Volume 2, Basic Applications , 2004 .

[25]  Amit Sahai,et al.  Secure Multi-Party Computation , 2013 .

[26]  Pascal Paillier,et al.  Public-Key Cryptosystems Based on Composite Degree Residuosity Classes , 1999, EUROCRYPT.

[27]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Dirk Fox,et al.  Advanced Encryption Standard (AES) , 1999, Datenschutz und Datensicherheit.

[29]  Shai Avidan,et al.  Blind Vision , 2006, ECCV.

[30]  Pong C. Yuen,et al.  On the Reconstruction of Face Images from Deep Face Templates , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Shai Avidan,et al.  Efficient Methods for Privacy Preserving Face Detection , 2006, NIPS.

[32]  Madhu Sudan,et al.  A Fuzzy Vault Scheme , 2006, Des. Codes Cryptogr..

[33]  Michael S. Ryoo,et al.  Privacy-Preserving Human Activity Recognition from Extreme Low Resolution , 2016, AAAI.

[34]  Chris Peikert,et al.  On Ideal Lattices and Learning with Errors over Rings , 2010, JACM.

[35]  Sudipta N. Sinha,et al.  Privacy Preserving Image Queries for Camera Localization , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[36]  Arun Ross,et al.  Visual Cryptography for Biometric Privacy , 2011, IEEE Transactions on Information Forensics and Security.

[37]  Stefanos Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Marc Pollefeys,et al.  Privacy Preserving Image-Based Localization , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Vishnu Naresh Boddeti.  Secure Face Matching Using Fully Homomorphic Encryption , 2018, 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[40]  Anil K. Jain,et al.  Learning a Fixed-Length Fingerprint Representation , 2019, IEEE transactions on pattern analysis and machine intelligence.

[41]  Simon Heron,et al.  Encryption: Advanced Encryption Standard (AES) , 2009 .

[42]  Thomas Brox,et al.  Inverting Visual Representations with Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  K. Srinathan,et al.  Blind Authentication: A Secure Crypto-Biometric Verification Protocol , 2010, IEEE Transactions on Information Forensics and Security.

[44]  Marina Blanton,et al.  Secure Multiparty Computation , 2011, Encyclopedia of Cryptography and Security.

[45]  Anil K. Jain,et al.  Handbook of Fingerprint Recognition , 2005, Springer Professional Computing.

[46]  Mauro Barni,et al.  Privacy Protection in Biometric-Based Recognition Systems: A marriage between cryptography and signal processing , 2015, IEEE Signal Processing Magazine.

[47]  K. Srinathan,et al.  Private Content Based Image Retrieval , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Anil K. Jain,et al.  Longitudinal study of fingerprint recognition , 2015, Proceedings of the National Academy of Sciences.

[49]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[50]  Yoichi Sato,et al.  Privacy-Preserving Visual Learning Using Doubly Permuted Homomorphic Encryption , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[51]  David J. Crandall,et al.  Enhancing Lifelogging Privacy by Detecting Screens , 2016, CHI.

[52]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[53]  Rama Chellappa,et al.  Cancelable Biometrics: A review , 2015, IEEE Signal Processing Magazine.

[54]  Shengcai Liao,et al.  Learning Face Representation from Scratch , 2014, ArXiv.

[55]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[56]  Mauro Barni,et al.  Encrypted signal processing for privacy protection: Conveying the utility of homomorphic encryption and multiparty computation , 2013, IEEE Signal Processing Magazine.

[57]  Nalini K. Ratha,et al.  Enhancing security and privacy in biometrics-based authentication systems , 2001, IBM Syst. J..

[58]  K. Srinathan,et al.  Efficient privacy preserving video surveillance , 2009, 2009 IEEE 12th International Conference on Computer Vision.