Falcon: Honest-Majority Maliciously Secure Framework for Private Deep Learning

Abstract We propose Falcon, an end-to-end 3-party protocol for efficient private training and inference of large machine learning models. Falcon presents four main advantages – (i) It is highly expressive with support for high capacity networks such as VGG16 (ii) it supports batch normalization which is important for training complex networks such as AlexNet (iii) Falcon guarantees security with abort against malicious adversaries, assuming an honest majority (iv) Lastly, Falcon presents new theoretical insights for protocol design that make it highly efficient and allow it to outperform existing secure deep learning solutions. Compared to prior art for private inference, we are about 8× faster than SecureNN (PETS’19) on average and comparable to ABY3 (CCS’18). We are about 16 − 200× more communication efficient than either of these. For private training, we are about 6× faster than SecureNN, 4.4× faster than ABY3 and about 2−60× more communication efficient. Our experiments in the WAN setting show that over large networks and datasets, compute operations dominate the overall latency of MPC, as opposed to the communication.

[1]  Yehuda Lindell,et al.  Information-theoretically secure protocols and security under composition , 2006, STOC '06.

[2]  Ran Canetti,et al.  Security and Composition of Multiparty Cryptographic Protocols , 2000, Journal of Cryptology.

[3]  Farinaz Koushanfar,et al.  XONN: XNOR-based Oblivious Deep Neural Network Inference , 2019, IACR Cryptol. ePrint Arch..

[4]  Bo Chen,et al.  Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Farinaz Koushanfar,et al.  Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications , 2018, IACR Cryptol. ePrint Arch..

[6]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[7]  Ivan Damgård,et al.  Multiparty Computation from Somewhat Homomorphic Encryption , 2012, IACR Cryptol. ePrint Arch..

[8]  Yao Lu,et al.  Oblivious Neural Network Predictions via MiniONN Transformations , 2017, IACR Cryptol. ePrint Arch..

[9]  Simon Haykin,et al.  GradientBased Learning Applied to Document Recognition , 2001 .

[10]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[11]  Raluca Ada Popa,et al.  Delphi: A Cryptographic Inference System for Neural Networks , 2020, IACR Cryptol. ePrint Arch..

[12]  Yehuda Lindell,et al.  Information-Theoretically Secure Protocols and Security under Composition , 2010, SIAM J. Comput..

[13]  Ivan Damgård,et al.  Semi-Homomorphic Encryption and Multiparty Computation , 2011, IACR Cryptol. ePrint Arch..

[14]  Matt J. Kusner,et al.  QUOTIENT: Two-Party Secure Neural Network Training and Prediction , 2019, CCS.

[15]  Arpita Patra,et al.  FLASH: Fast and Robust Framework for Privacy-preserving Machine Learning , 2020, IACR Cryptol. ePrint Arch..

[16]  Jiayu Wu,et al.  Tiny ImageNet Challenge , 2017 .

[17]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[18]  Peter Rindal,et al.  ABY3: A Mixed Protocol Framework for Machine Learning , 2018, IACR Cryptol. ePrint Arch..

[19]  Marcel Keller,et al.  Secure Evaluation of Quantized Neural Networks , 2019, IACR Cryptol. ePrint Arch..

[20]  Frederik Vercauteren,et al.  EPIC: Efficient Private Image Classification (or: Learning from the Masters) , 2019, CT-RSA.

[21]  Michael Zohner,et al.  ABY - A Framework for Efficient Mixed-Protocol Secure Two-Party Computation , 2015, NDSS.

[22]  Somesh Jha,et al.  Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures , 2015, CCS.

[23]  David J. Wu,et al.  Secure genome-wide association analysis using multiparty computation , 2018, Nature Biotechnology.

[24]  Silvio Micali,et al.  A Completeness Theorem for Protocols with Honest Majority , 1987, STOC 1987.

[25]  Ajith Suresh,et al.  Trident: Efficient 4PC Framework for Privacy Preserving Machine Learning , 2019, IACR Cryptol. ePrint Arch..

[26]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[27]  Sameer Wagh,et al.  SecureNN: 3-Party Secure Computation for Neural Network Training , 2019, Proc. Priv. Enhancing Technol..

[28]  Payman Mohassel,et al.  SecureML: A System for Scalable Privacy-Preserving Machine Learning , 2017, 2017 IEEE Symposium on Security and Privacy (SP).

[29]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[30]  Vitaly Shmatikov,et al.  Membership Inference Attacks Against Machine Learning Models , 2016, 2017 IEEE Symposium on Security and Privacy (SP).

[31]  Yehuda Lindell,et al.  High-Throughput Semi-Honest Secure Three-Party Computation with an Honest Majority , 2016, IACR Cryptol. ePrint Arch..

[32]  Shafi Goldwasser,et al.  Machine Learning Classification over Encrypted Data , 2015, NDSS.

[33]  Yehuda Lindell,et al.  High-Throughput Secure Three-Party Computation for Malicious Adversaries and an Honest Majority , 2017, IACR Cryptol. ePrint Arch..

[34]  Elie Bursztein,et al.  Rethinking the Detection of Child Sexual Abuse Imagery on the Internet , 2019, WWW.

[35]  Melek Önen,et al.  FHE-Compatible Batch Normalization for Privacy Preserving Deep Learning , 2018, DPM/CBT@ESORICS.

[36]  Constance Morel,et al.  Privacy-Preserving Classification on Deep Neural Network , 2017, IACR Cryptol. ePrint Arch..

[37]  Nishant Kumar,et al.  CrypTFlow: Secure TensorFlow Inference , 2020, 2020 IEEE Symposium on Security and Privacy (SP).

[38]  Yehuda Lindell,et al.  Privacy Preserving Data Mining , 2000, Journal of Cryptology.

[39]  Raluca Ada Popa,et al.  Delphi: A Cryptographic Inference System for Neural Networks , 2020 .

[40]  Farinaz Koushanfar,et al.  DeepSecure: Scalable Provably-Secure Deep Learning , 2017, 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC).

[41]  Adi Shamir,et al.  How to share a secret , 1979, CACM.

[42]  Octavian Catrina,et al.  Secure Computation with Fixed-Point Numbers , 2010, Financial Cryptography.

[43]  Yihua Zhang,et al.  Secure Computation on Floating Point Numbers , 2013, NDSS.

[44]  Andrew Chi-Chih Yao,et al.  Protocols for secure computations , 1982, FOCS 1982.

[45]  Michael Naehrig,et al.  CryptoNets: applying neural networks to encrypted data with high throughput and accuracy , 2016, ICML 2016.

[46]  Silvio Micali,et al.  How to play ANY mental game , 1987, STOC.

[47]  Li Fei-Fei,et al.  Faster CryptoNets: Leveraging Sparsity for Real-World Encrypted Inference , 2018, ArXiv.

[48]  Ran Canetti,et al.  Universally composable security: a new paradigm for cryptographic protocols , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[49]  Yuval Ishai,et al.  Zero-Knowledge Proofs on Secret-Shared Data via Fully Linear PCPs , 2019, CRYPTO.

[50]  Fan Zhang,et al.  Stealing Machine Learning Models via Prediction APIs , 2016, USENIX Security Symposium.

[51]  Yuval Ishai,et al.  Practical Fully Secure Three-Party Computation via Sublinear Distributed Zero-Knowledge Proofs , 2019, CCS.

[52]  Brent Waters,et al.  A Framework for Efficient and Composable Oblivious Transfer , 2008, CRYPTO.

[53]  Aseem Rastogi,et al.  EzPC: Programmable, Efficient, and Scalable Secure Two-Party Computation , 2018, IACR Cryptol. ePrint Arch..

[54]  Ion Stoica,et al.  Helen: Maliciously Secure Coopetitive Learning for Linear Models , 2019, 2019 IEEE Symposium on Security and Privacy (SP).

[55]  Hassan Takabi,et al.  CryptoDL: Deep Neural Networks over Encrypted Data , 2017, ArXiv.

[56]  Dan Bogdanov,et al.  Sharemind: A Framework for Fast Privacy-Preserving Computations , 2008, ESORICS.

[57]  Hassan Takabi,et al.  Deep Neural Networks Classification over Encrypted Data , 2019, CODASPY.

[58]  Anantha Chandrakasan,et al.  Gazelle: A Low Latency Framework for Secure Neural Network Inference , 2018, IACR Cryptol. ePrint Arch..

[59]  Arpita Patra,et al.  BLAZE: Blazing Fast Privacy-Preserving Machine Learning , 2020, IACR Cryptol. ePrint Arch..

[60]  Ashish Choudhury,et al.  ASTRA: High Throughput 3PC over Rings with Application to Secure Prediction , 2019, IACR Cryptol. ePrint Arch..