HERS: Homomorphically Encrypted Representation Search

We present a method to search for a probe (or query) image representation against a large gallery in the encrypted domain. We require that the probe and gallery images be represented in terms of a fixed-length representation, which is typical for representations obtained from learned networks. Our encryption scheme is agnostic to how the fixed-length representation is obtained and can, therefore, be applied to any fixed-length representation in any application domain. Our method, dubbed HERS (Homomorphically Encrypted Representation Search), operates by (i) compressing the representation towards its estimated intrinsic dimensionality, (ii) encrypting the compressed representation using the proposed fully homomorphic encryption scheme, and (iii) searching against a gallery of encrypted representations directly in the encrypted domain, without decrypting them, and with minimal loss of accuracy. Numerical results on large galleries of face, fingerprint, and object datasets such as ImageNet show that, for the first time, accurate and fast image search within the encrypted domain is feasible at scale (296 seconds; 46x speedup over state-of-the-art for face search against a background of 1 million).

[1]  Mauro Barni,et al.  Privacy Protection in Biometric-Based Recognition Systems: A marriage between cryptography and signal processing , 2015, IEEE Signal Processing Magazine.

[2]  Patrick Pérez,et al.  Reconstructing an image from its local descriptors , 2011, CVPR 2011.

[3]  Oded Goldreich,et al.  Foundations of Cryptography: Volume 2, Basic Applications , 2004 .

[4]  Stefanos Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Dirk Fox,et al.  Advanced Encryption Standard (AES) , 1999, Datenschutz und Datensicherheit.

[6]  Yong Jae Lee,et al.  Learning to Anonymize Faces for Privacy Preserving Action Detection , 2018, ECCV.

[7]  Fernando Pérez-González,et al.  Fully Private Noninteractive Face Verification , 2013, IEEE Transactions on Information Forensics and Security.

[8]  Jung Hee Cheon,et al.  Ghostshell: Secure Biometric Authentication using Integrity-based Homomorphic Evaluations , 2016, IACR Cryptol. ePrint Arch..

[9]  Anil K. Jain,et al.  Learning a Fixed-Length Fingerprint Representation , 2019, IEEE transactions on pattern analysis and machine intelligence.

[10]  Chris Peikert,et al.  On Ideal Lattices and Learning with Errors over Rings , 2010, JACM.

[11]  Kang Ryoung Park,et al.  A New Method for Generating an Invariant Iris Private Key Based on the Fuzzy Vault System , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[13]  Sing Bing Kang,et al.  Revealing Scenes by Inverting Structure From Motion Reconstructions , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Yoichi Sato,et al.  Privacy-Preserving Visual Learning Using Doubly Permuted Homomorphic Encryption , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[15]  Ann Cavoukian,et al.  Biometric Encryption , 2011, Encyclopedia of Cryptography and Security.

[16]  Shai Avidan,et al.  Efficient Methods for Privacy Preserving Face Detection , 2006, NIPS.

[17]  K. Srinathan,et al.  Private Content Based Image Retrieval , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Ross B. Girshick,et al.  Reducing Overfitting in Deep Networks by Decorrelating Representations , 2015, ICLR.

[19]  Sharath Pankanti,et al.  Fuzzy Vault for Fingerprints , 2005, AVBPA.

[20]  Anil K. Jain,et al.  Hardening Fingerprint Fuzzy Vault Using Password , 2007, ICB.

[21]  David J. Crandall,et al.  Enhancing Lifelogging Privacy by Detecting Screens , 2016, CHI.

[22]  Thomas Brox,et al.  Inverting Visual Representations with Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  K. Srinathan,et al.  Blind Authentication: A Secure Crypto-Biometric Verification Protocol , 2010, IEEE Transactions on Information Forensics and Security.

[24]  K. Srinathan,et al.  Efficient privacy preserving video surveillance , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[25]  Pong C. Yuen,et al.  On the Reconstruction of Face Images from Deep Face Templates , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[27]  Mauro Barni,et al.  Encrypted signal processing for privacy protection: Conveying the utility of homomorphic encryption and multiparty computation , 2013, IEEE Signal Processing Magazine.

[28]  Michael Naehrig,et al.  CryptoNets: applying neural networks to encrypted data with high throughput and accuracy , 2016, ICML 2016.

[29]  Moni Naor,et al.  Visual Cryptography , 1994, Encyclopedia of Multimedia.

[30]  Marc Pollefeys,et al.  Privacy Preserving Image-Based Localization , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Michael S. Ryoo,et al.  Privacy-Preserving Human Activity Recognition from Extreme Low Resolution , 2016, AAAI.

[32]  Sudipta N. Sinha,et al.  Privacy Preserving Image Queries for Camera Localization , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[33]  Arun Ross,et al.  Visual Cryptography for Biometric Privacy , 2011, IEEE Transactions on Information Forensics and Security.

[34]  Amit Sahai,et al.  Secure Multi-Party Computation , 2013 .

[35]  Simon Heron,et al.  Encryption: Advanced Encryption Standard (AES) , 2009 .

[36]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[37]  Antonio Torralba,et al.  HOGgles: Visualizing Object Detection Features , 2013, 2013 IEEE International Conference on Computer Vision.

[38]  Vishnu Naresh Boddeti,et al.  On the Intrinsic Dimensionality of Image Representations , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  B. V. K. Vijaya Kumar,et al.  A Framework for Binding and Retrieving Class-Specific Information to and from Image Patterns Using Correlation Filters , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Anantha Chandrakasan,et al.  Gazelle: A Low Latency Framework for Secure Neural Network Inference , 2018, IACR Cryptol. ePrint Arch..

[41]  Ira Kemelmacher-Shlizerman,et al.  The MegaFace Benchmark: 1 Million Faces for Recognition at Scale , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Rama Chellappa,et al.  Cancelable Biometrics: A review , 2015, IEEE Signal Processing Magazine.

[43]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[44]  Shai Avidan,et al.  Blind Vision , 2006, ECCV.

[45]  Madhu Sudan,et al.  A Fuzzy Vault Scheme , 2006, Des. Codes Cryptogr..

[46]  Frederik Vercauteren,et al.  Somewhat Practical Fully Homomorphic Encryption , 2012, IACR Cryptol. ePrint Arch..

[47]  Vishnu Naresh Boddeti.  Secure Face Matching Using Fully Homomorphic Encryption , 2018, 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[48]  Nalini K. Ratha,et al.  Enhancing security and privacy in biometrics-based authentication systems , 2001, IBM Syst. J..

[49]  Oded Goldreich,et al.  The Foundations of Cryptography - Volume 2: Basic Applications , 2001 .

[50]  Tatsuya Harada,et al.  Image Reconstruction from Bag-of-Visual-Words , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Anil K. Jain,et al.  Handbook of Fingerprint Recognition , 2005, Springer Professional Computing.

[52]  Craig Gentry,et al.  Implementing Gentry's Fully-Homomorphic Encryption Scheme , 2011, EUROCRYPT.

[53]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[54]  Pascal Paillier,et al.  Public-Key Cryptosystems Based on Composite Degree Residuosity Classes , 1999, EUROCRYPT.

[55]  Shengcai Liao,et al.  Learning Face Representation from Scratch , 2014, ArXiv.

[56]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Marina Blanton,et al.  Secure Multiparty Computation , 2011, Encyclopedia of Cryptography and Security.

[58]  Anil K. Jain,et al.  Longitudinal study of fingerprint recognition , 2015, Proceedings of the National Academy of Sciences.