CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy

Applying machine learning to a problem which involves medical, financial, or other types of sensitive data, not only requires accurate predictions but also careful attention to maintaining data privacy and security. Legal and ethical requirements may prevent the use of cloud-based machine learning solutions for such tasks. In this work, we will present a method to convert learned neural networks to CryptoNets, neural networks that can be applied to encrypted data. This allows a data owner to send their data in an encrypted form to a cloud service that hosts the network. The encryption ensures that the data remains confidential since the cloud does not have access to the keys needed to decrypt it. Nevertheless, we will show that the cloud service is capable of applying the neural network to the encrypted data to make encrypted predictions, and also return them in encrypted form. These encrypted predictions can be sent back to the owner of the secret key who can decrypt them. Therefore, the cloud service does not gain any information about the raw data nor about the prediction it made. We demonstrate CryptoNets on the MNIST optical character recognition tasks. CryptoNets achieve 99% accuracy and can make around 59000 predictions per hour on a single PC. Therefore, they allow high throughput, accurate, and private predictions.

[1]  Michael Naehrig,et al.  Manual for Using Homomorphic Encryption for Bioinformatics , 2017, Proceedings of the IEEE.

[2]  Louis J. M. Aslett,et al.  Encrypted statistical machine learning: new privacy preserving methods , 2015, ArXiv.

[3]  Louis J. M. Aslett,et al.  A review of homomorphic encryption and software tools for encrypted statistical machine learning , 2015, ArXiv.

[4]  Pengtao Xie,et al.  Crypto-Nets: Neural Networks over Encrypted Data , 2014, ArXiv.

[5]  Roi Livni,et al.  On the Computational Efficiency of Training Neural Networks , 2014, NIPS.

[6]  Zvika Brakerski,et al.  Efficient Fully Homomorphic Encryption from (Standard) $\mathsf{LWE}$ , 2014, SIAM J. Comput..

[7]  Michael Naehrig,et al.  Improved Security for a Ring-Based Fully Homomorphic Encryption Scheme , 2013, IMACC.

[8]  M. Prabhakaran,et al.  Secure Multi-Party Computation , 2013, Secure Multi-Party Computation.

[9]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[10]  Michael Naehrig,et al.  ML Confidential: Machine Learning on Encrypted Data , 2012, ICISC.

[11]  Craig Gentry,et al.  Homomorphic Evaluation of the AES Circuit , 2012, IACR Cryptol. ePrint Arch..

[12]  Vinod Vaikuntanathan,et al.  On-the-fly multiparty computation on the cloud via multikey fully homomorphic encryption , 2012, STOC '12.

[13]  Craig Gentry,et al.  Fully Homomorphic Encryption with Polylog Overhead , 2012, EUROCRYPT.

[14]  Dong Yu,et al.  Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[15]  Vinod Vaikuntanathan,et al.  Efficient Fully Homomorphic Encryption from (Standard) LWE , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[16]  Vinod Vaikuntanathan,et al.  Can homomorphic encryption be practical? , 2011, CCSW '11.

[17]  Ron Steinfeld,et al.  Making NTRU as Secure as Worst-Case Problems over Ideal Lattices , 2011, EUROCRYPT.

[18]  Sheng Zhong,et al.  Privacy-Preserving Backpropagation Neural Network Learning , 2009, IEEE Transactions on Neural Networks.

[19]  Craig Gentry,et al.  Fully homomorphic encryption using ideal lattices , 2009, STOC '09.

[20]  Mauro Barni,et al.  Enhancing Privacy in Remote Data Classification , 2008, SEC.

[21]  Mikhail J. Atallah,et al.  Efficient Privacy-Preserving k-Nearest Neighbor Search , 2008, 2008 The 28th International Conference on Distributed Computing Systems.

[22]  Mauro Barni,et al.  Oblivious Neural Network Computing via Homomorphic Encryption , 2007, EURASIP J. Inf. Secur..

[23]  Mauro Barni,et al.  A privacy-preserving protocol for neural-network-based computation , 2006, MM&Sec '06.

[24]  Cynthia Dwork,et al.  Differential Privacy , 2006, ICALP.

[25]  Ramakrishnan Srikant,et al.  Privacy-preserving data mining , 2000, SIGMOD '00.

[26]  D. Eisenbud.  Commutative Algebra: with a View Toward Algebraic Geometry , 1995 .

[27]  Zhang Wen-k.  Secure Multi-party Computation , 2014 .

[28]  Stan Matwin,et al.  Privacy Preserving K-nearest Neighbor Classification , 2005, Int. J. Netw. Secur..

[29]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[30]  Ronald L. Rivest,et al.  ON DATA BANKS AND PRIVACY HOMOMORPHISMS , 1978 .