Privacy-preserving remote deep-learning-based inference under constrained client-side environment

Remote deep learning paradigm raises important privacy concerns related to clients sensitive data and deep learning models. However, dealing with such concerns may come at the expense of more client-side overhead, which does not fit applications relying on constrained environments. In this paper, we propose a privacy-preserving solution for deep-learning-based inference, which ensures effectiveness and privacy, while meeting efficiency requirements of constrained client-side environments. The solution adopts the non-colluding two-server architecture, which prevents accuracy loss as it avoids using approximation of activation functions, and copes with constrained client-side due to low overhead cost. The solution also ensures privacy by leveraging two reversible perturbation techniques in combination with paillier homomorphic encryption scheme. Client-side overhead evaluation compared to the conventional homomorphic encryption approach, achieves up to more than two thousands times improvement in terms of execution time, and up to more than thirty times improvement in terms of the transmitted data size.

[1]  Kousha Etessami,et al.  Recursive Markov chains, stochastic grammars, and monotone systems of nonlinear equations , 2005, JACM.

[2]  Pascal Paillier,et al.  Paillier Encryption and Signature Schemes , 2005, Encyclopedia of Cryptography and Security.

[3]  Yurong Liu,et al.  A survey of deep neural network architectures and their applications , 2017, Neurocomputing.

[4]  Said El Hajji,et al.  Hybrid Homomorphic Encryption Method for Protecting the Privacy of Banking Data in the Cloud , 2015 .

[5]  Yehuda Lindell,et al.  Introduction to Modern Cryptography, Second Edition , 2014 .

[6]  Suat Ozdemir,et al.  Routing in Fog-Enabled IoT Platforms: A Survey and an SDN-Based Solution , 2018, IEEE Internet of Things Journal.

[7]  Mauro Conti,et al.  A Survey on Homomorphic Encryption Schemes , 2017, ACM Comput. Surv..

[8]  Byunghan Lee,et al.  Deep learning in bioinformatics , 2016, Briefings Bioinform..

[9]  Joseph H. Silverman,et al.  NTRU: A Ring-Based Public Key Cryptosystem , 1998, ANTS.

[10]  Gregory D. Hager,et al.  Temporal Convolutional Networks for Action Segmentation and Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Chris Peikert,et al.  Better Key Sizes (and Attacks) for LWE-Based Encryption , 2011, CT-RSA.

[12]  Abdelouahid Derhab,et al.  Deep learning approaches for anomaly-based intrusion detection systems: A survey, taxonomy, and open issues , 2020, Knowl. Based Syst..

[13]  Vitaly Shmatikov,et al.  Privacy-preserving deep learning , 2015, Allerton.

[14]  Boaz Tsaban,et al.  Cryptanalysis of the MORE symmetric key fully homomorphic encryption scheme , 2015, J. Math. Cryptol..

[15]  Abdelouahid Derhab,et al.  A review of privacy-preserving techniques for deep learning , 2020, Neurocomputing.

[16]  Jian Weng,et al.  A Comprehensive Survey on Secure Outsourced Computation and Its Applications , 2019, IEEE Access.

[17]  Hassan Takabi,et al.  CryptoDL: Deep Neural Networks over Encrypted Data , 2017, ArXiv.

[18]  Guang-Zhong Yang,et al.  Deep Learning for Health Informatics , 2017, IEEE Journal of Biomedical and Health Informatics.

[19]  Richard Nock,et al.  Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption , 2017, ArXiv.

[20]  R. Brualdi Combinatorial Matrix Classes , 2006 .

[21]  Oded Regev,et al.  On lattices, learning with errors, random linear codes, and cryptography , 2009, JACM.

[22]  Craig Gentry,et al.  (Leveled) Fully Homomorphic Encryption without Bootstrapping , 2014, ACM Trans. Comput. Theory.

[23]  Hassan Takabi,et al.  Privacy-preserving Machine Learning in Cloud , 2017, CCSW.

[24]  Le Trieu Phong,et al.  Privacy-Preserving Deep Learning for any Activation Function , 2018, ArXiv.

[25]  Xixiang Lv,et al.  2P-DNN : Privacy-Preserving Deep Neural Networks Based on Homomorphic Cryptosystem , 2018, ArXiv.

[26]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[27]  Said El Hajji,et al.  Homomorphic Encryption Applied to the Cloud Computing Security , 2012 .

[28]  Ivan Damgård,et al.  A Generalisation, a Simplification and Some Applications of Paillier's Probabilistic Public-Key System , 2001, Public Key Cryptography.

[29]  Harry Chandra Tanuwidjaja,et al.  Limitations of Privacy-Preserving for Confidential Data Training by Deep Learning , 2018 .

[30]  Michael Naehrig,et al.  Improved Security for a Ring-Based Fully Homomorphic Encryption Scheme , 2013, IMACC.

[31]  Farinaz Koushanfar,et al.  DeepSecure: Scalable Provably-Secure Deep Learning , 2017, 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC).

[32]  Constance Morel,et al.  Privacy-Preserving Classification on Deep Neural Network , 2017, IACR Cryptol. ePrint Arch..

[33]  Julian Jang,et al.  Towards privacy-preserving classification in neural networks , 2016, 2016 14th Annual Conference on Privacy, Security and Trust (PST).

[34]  Dan Boneh,et al.  Evaluating 2-DNF Formulas on Ciphertexts , 2005, TCC.

[35]  Li Deng,et al.  A tutorial survey of architectures, algorithms, and applications for deep learning , 2014, APSIPA Transactions on Signal and Information Processing.

[36]  Cong Wang,et al.  GELU-Net: A Globally Encrypted, Locally Unencrypted Deep Neural Network for Privacy-Preserved Learning , 2018, IJCAI.

[37]  Michael Naehrig,et al.  CryptoNets: applying neural networks to encrypted data with high throughput and accuracy , 2016, ICML 2016.

[38]  Mani B. Srivastava,et al.  CryptoImg: Privacy preserving processing over encrypted images , 2016, 2016 IEEE Conference on Communications and Network Security (CNS).

[39]  T. Elgamal A public key cryptosystem and a signature scheme based on discrete logarithms , 1984, CRYPTO 1984.

[40]  Mohamed Amine Ferrag,et al.  Deep learning for cyber security intrusion detection: Approaches, datasets, and comparative study , 2020, J. Inf. Secur. Appl..

[41]  Andreas Kamilaris,et al.  Deep learning in agriculture: A survey , 2018, Comput. Electron. Agric..

[42]  Peng Cheng,et al.  Lightweight and Unobtrusive Data Obfuscation at IoT Edge for Remote Inference , 2019, IEEE Internet of Things Journal.

[43]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[44]  Yehuda Lindell,et al.  Introduction to Modern Cryptography , 2004 .

[45]  Zhikui Chen,et al.  Privacy Preserving Back-Propagation Based on BGV on Cloud , 2015, 2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium on Cyberspace Safety and Security, and 2015 IEEE 12th International Conference on Embedded Software and Systems.

[46]  Claudio Bettini,et al.  The Privacy Implications of Cyber Security Systems , 2018, ACM Comput. Surv..

[47]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[48]  Lucian Mihai Itu,et al.  Towards Privacy-Preserving Deep Learning based Medical Imaging Applications , 2019, 2019 IEEE International Symposium on Medical Measurements and Applications (MeMeA).

[49]  Sherman S. M. Chow,et al.  Optimizing Privacy-Preserving Outsourced Convolutional Neural Network Predictions , 2020, IEEE Transactions on Dependable and Secure Computing.