CrypTFlow2: Practical 2-Party Secure Inference

We present CrypTFlow2, a cryptographic framework for secure inference over realistic Deep Neural Networks (DNNs) using secure 2-party computation. CrypTFlow2 protocols are both correct -- i.e., their outputs are bitwise equivalent to the cleartext execution -- and efficient -- they outperform the state-of-the-art protocols in both latency and scale. At the core of CrypTFlow2, we have new 2PC protocols for secure comparison and division, designed carefully to balance round and communication complexity for secure inference tasks. Using CrypTFlow2, we present the first secure inference over ImageNet-scale DNNs like ResNet50 and DenseNet121. These DNNs are at least an order of magnitude larger than those considered in the prior work of 2-party DNN inference. Even on the benchmarks considered by prior work, CrypTFlow2 requires an order of magnitude less communication and 20x-30x less time than the state-of-the-art.

[1]  Mihir Bellare,et al.  Efficient Garbling from a Fixed-Key Blockcipher , 2013, 2013 IEEE Symposium on Security and Privacy.

[2]  Yehuda Lindell,et al.  How To Simulate It - A Tutorial on the Simulation Proof Technique , 2016, IACR Cryptol. ePrint Arch..

[3]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[4]  Ran Canetti,et al.  Security and Composition of Multiparty Cryptographic Protocols , 2000, Journal of Cryptology.

[5]  Donald Beaver,et al.  Correlated pseudorandomness and the complexity of private computations , 1996, STOC '96.

[6]  Michael Zohner,et al.  ABY - A Framework for Efficient Mixed-Protocol Secure Two-Party Computation , 2015, NDSS.

[7]  Thomas Schneider,et al.  Improved Multiplication Triple Generation over Rings via RLWE-based AHE , 2019, IACR Cryptol. ePrint Arch..

[8]  Zvika Brakerski,et al.  Fully Homomorphic Encryption without Modulus Switching from Classical GapSVP , 2012, CRYPTO.

[9]  Bo Chen,et al.  Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Jonathan Katz,et al.  Efficient and Secure Multiparty Computation from Fixed-Key Block Ciphers , 2020, 2020 IEEE Symposium on Security and Privacy (SP).

[11]  Alex J. Malozemoff,et al.  Efficiently Enforcing Input Validity in Secure Two-party Computation , 2016, IACR Cryptol. ePrint Arch..

[12]  Farinaz Koushanfar,et al.  XONN: XNOR-based Oblivious Deep Neural Network Inference , 2019, IACR Cryptol. ePrint Arch..

[13]  Peter Rindal,et al.  ABY3: A Mixed Protocol Framework for Machine Learning , 2018, IACR Cryptol. ePrint Arch..

[14]  Yixing Lao,et al.  nGraph-HE: a graph compiler for deep learning on homomorphically encrypted data , 2018, IACR Cryptol. ePrint Arch..

[15]  Craig Gentry,et al.  Fully homomorphic encryption using ideal lattices , 2009, STOC '09.

[16]  Tal Malkin,et al.  Garbled Neural Networks are Practical , 2019, IACR Cryptol. ePrint Arch..

[17]  Geoffroy Couteau,et al.  New Protocols for Secure Equality Test and Comparison , 2018, ACNS.

[18]  Yao Lu,et al.  Oblivious Neural Network Predictions via MiniONN Transformations , 2017, IACR Cryptol. ePrint Arch..

[19]  Adi Shamir,et al.  How to share a secret , 1979, CACM.

[20]  Matt J. Kusner,et al.  QUOTIENT: Two-Party Secure Neural Network Training and Prediction , 2019, CCS.

[21]  Rosario Cammarota,et al.  nGraph-HE2: A High-Throughput Framework for Neural Network Inference on Encrypted Data , 2019, IACR Cryptol. ePrint Arch..

[22]  Raluca Ada Popa,et al.  Delphi: A Cryptographic Inference System for Neural Networks , 2020, IACR Cryptol. ePrint Arch..

[23]  Andrew Chi-Chih Yao,et al.  How to generate and exchange secrets , 1986, 27th Annual Symposium on Foundations of Computer Science (sfcs 1986).

[24]  Farinaz Koushanfar,et al.  Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications , 2018, IACR Cryptol. ePrint Arch..

[25]  Donald Beaver,et al.  Efficient Multiparty Protocols Using Circuit Randomization , 1991, CRYPTO.

[26]  Frederik Vercauteren,et al.  Somewhat Practical Fully Homomorphic Encryption , 2012, IACR Cryptol. ePrint Arch..

[27]  Ahmad-Reza Sadeghi,et al.  Pushing the Communication Barrier in Secure Computation using Lookup Tables , 2017, NDSS.

[28]  Gilles Brassard,et al.  All-or-Nothing Disclosure of Secrets , 1986, CRYPTO.

[29]  Anantha Chandrakasan,et al.  Gazelle: A Low Latency Framework for Secure Neural Network Inference , 2018, IACR Cryptol. ePrint Arch..

[30]  Hao Chen,et al.  CHET: an optimizing compiler for fully-homomorphic neural-network inferencing , 2019, PLDI.

[31]  Marcel Keller,et al.  Improved Primitives for MPC over Mixed Arithmetic-Binary Circuits , 2020, IACR Cryptol. ePrint Arch..

[32]  Yuval Ishai,et al.  LevioSA: Lightweight Secure Arithmetic Computation , 2019, CCS.

[33]  Juan A. Garay,et al.  Practical and Secure Solutions for Integer Comparison , 2007, Public Key Cryptography.

[34]  Markus Nagel,et al.  Data-Free Quantization Through Weight Equalization and Bias Correction , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[35]  Payman Mohassel,et al.  SecureML: A System for Scalable Privacy-Preserving Machine Learning , 2017, 2017 IEEE Symposium on Security and Privacy (SP).

[36]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Nishant Kumar,et al.  CrypTFlow: Secure TensorFlow Inference , 2020, 2020 IEEE Symposium on Security and Privacy (SP).

[38]  Vladimir Kolesnikov,et al.  Improved OT Extension for Transferring Short Secrets , 2013, CRYPTO.

[39]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[40]  Sameer Wagh,et al.  SecureNN: 3-Party Secure Computation for Neural Network Training , 2019, Proc. Priv. Enhancing Technol..

[41]  Michael O. Rabin,et al.  How To Exchange Secrets with Oblivious Transfer , 2005, IACR Cryptol. ePrint Arch..

[42]  Stefan Katzenbeisser,et al.  HyCC: Compilation of Hybrid Protocols for Practical Secure Computation , 2018, CCS.

[43]  Hao Chen,et al.  Simple Encrypted Arithmetic Library v2.3.0 , 2017 .

[44]  Ran El-Yaniv,et al.  Binarized Neural Networks , 2016, ArXiv.

[45]  Frederik Vercauteren,et al.  Fully homomorphic SIMD operations , 2012, Designs, Codes and Cryptography.

[46]  Yehuda Lindell,et al.  Fast Garbling of Circuits Under Standard Assumptions , 2015, Journal of Cryptology.

[47]  Aseem Rastogi,et al.  EzPC: Programmable and Efficient Secure Two-Party Computation for Machine Learning , 2019, 2019 IEEE European Symposium on Security and Privacy (EuroS&P).

[48]  Marcel Keller,et al.  Secure Evaluation of Quantized Neural Networks , 2019, IACR Cryptol. ePrint Arch..

[49]  Yehuda Lindell,et al.  More efficient oblivious transfer and extensions for faster secure computation , 2013, CCS.

[50]  Farinaz Koushanfar,et al.  DeepSecure: Scalable Provably-Secure Deep Learning , 2017, 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC).

[51]  Oded Goldreich,et al.  A randomized protocol for signing contracts , 1985, CACM.

[52]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[53]  Ion Stoica,et al.  Helen: Maliciously Secure Coopetitive Learning for Linear Models , 2019, 2019 IEEE Symposium on Security and Privacy (SP).

[54]  Mariana Raykova,et al.  Secure Computation for Machine Learning With SPDZ , 2019, ArXiv.

[55]  Silvio Micali,et al.  A Completeness Theorem for Protocols with Honest Majority , 1987, STOC 1987.

[56]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Shafi Goldwasser,et al.  Machine Learning Classification over Encrypted Data , 2015, NDSS.

[58]  Murat Kantarcioglu,et al.  CheapSMC: A Framework to Minimize Secure Multiparty Computation Cost in the Cloud , 2016, DBSec.

[59]  Michael Naehrig,et al.  CryptoNets: applying neural networks to encrypted data with high throughput and accuracy , 2016, ICML 2016.

[60]  G. R. Blakley,et al.  Safeguarding cryptographic keys , 1899, 1979 International Workshop on Managing Requirements Knowledge (MARK).

[61]  Yuval Ishai,et al.  Extending Oblivious Transfers Efficiently , 2003, CRYPTO.