MUSE: Secure Inference Resilient to Malicious Clients

The increasing adoption of machine learning inference in applications has led to a corresponding increase in concerns surrounding the privacy guarantees offered by existing mechanisms for inference. Such concerns have motivated the construction of efficient secure inference protocols that allow parties to perform inference without revealing their sensitive information. Recently, there has been a proliferation of such proposals, rapidly improving efficiency. However, most of these protocols assume that the client is semi-honest, that is, the client does not deviate from the protocol; yet in practice, clients are many, have varying incentives, and can behave arbitrarily. To demonstrate that a malicious client can completely break the security of semi-honest protocols, we first develop a new model-extraction attack against many state-of-the-art secure inference protocols. Our attack enables a malicious client to learn model weights with 22×–312× fewer queries than the best black-box model-extraction attack [CJM20] and scales to much deeper networks. Motivated by the severity of our attack, we design and implement MUSE, an efficient two-party secure inference protocol resilient to malicious clients. MUSE introduces a novel cryptographic protocol for conditional disclosure of secrets to switch between authenticated additive secret shares and garbled circuit labels, and an improved Beaver’s triple generation procedure which is 8×–12.5× faster than existing techniques. These protocols allow MUSE to push a majority of its cryptographic overhead into a preprocessing phase: compared to the equivalent semi-honest protocol (which is close to state-ofthe-art), MUSE’s online phase is only 1.7×–2.2× slower and uses 1.4× more communication. Overall, MUSE is 13.4×– 21× faster and uses 2×–3.6× less communication than existing secure inference protocols which defend against malicious clients.

[1]  TU Dresden mhaehnel High-Resolution Side Channels for Untrusted Operating Systems , 2017 .

[2]  Tal Malkin,et al.  Garbled Neural Networks are Practical , 2019, IACR Cryptol. ePrint Arch..

[3]  Lucy Rosenbloom arXiv , 2019, The Charleston Advisor.

[4]  Dragos Rotaru,et al.  MArBled Circuits: Mixing Arithmetic and Boolean Circuits with Active Security , 2019, IACR Cryptol. ePrint Arch..

[5]  David Berthelot,et al.  High Accuracy and High Fidelity Extraction of Neural Networks , 2020, USENIX Security Symposium.

[6]  Peter Rindal,et al.  ABY3: A Mixed Protocol Framework for Machine Learning , 2018, IACR Cryptol. ePrint Arch..

[7]  Yang Zhang,et al.  MLCapsule: Guarded Offline Deployment of Machine Learning as a Service , 2018, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Yehuda Lindell,et al.  An Efficient Protocol for Secure Two-Party Computation in the Presence of Malicious Adversaries , 2007, Journal of Cryptology.

[9]  Yehuda Lindell,et al.  An Efficient Protocol for Secure Two-Party Computation in the Presence of Malicious Adversaries , 2007, EUROCRYPT.

[10]  Yehuda Lindell,et al.  A Proof of Security of Yao’s Protocol for Two-Party Computation , 2009, Journal of Cryptology.

[11]  Samuel Marchal,et al.  PRADA: Protecting Against DNN Model Stealing Attacks , 2018, 2019 IEEE European Symposium on Security and Privacy (EuroS&P).

[12]  Michael Naehrig,et al.  CryptoNets: applying neural networks to encrypted data with high throughput and accuracy , 2016, ICML 2016.

[13]  Marcel Keller,et al.  Actively Secure OT Extension with Optimal Overhead , 2015, CRYPTO.

[14]  Hassan Takabi,et al.  CryptoDL: Deep Neural Networks over Encrypted Data , 2017, ArXiv.

[15]  Pascal Paillier,et al.  Fast Homomorphic Evaluation of Deep Discretized Neural Networks , 2018, IACR Cryptol. ePrint Arch..

[16]  Hao Chen,et al.  CHET: an optimizing compiler for fully-homomorphic neural-network inferencing , 2019, PLDI.

[17]  Stefan Mangard,et al.  Malware Guard Extension: Using SGX to Conceal Cache Attacks , 2017, DIMVA.

[18]  David Rolnick,et al.  Reverse-engineering deep ReLU networks , 2019, ICML.

[19]  Yoshua Bengio,et al.  BinaryConnect: Training Deep Neural Networks with binary weights during propagations , 2015, NIPS.

[20]  Aseem Rastogi,et al.  EzPC: Programmable and Efficient Secure Two-Party Computation for Machine Learning , 2019, 2019 IEEE European Symposium on Security and Privacy (EuroS&P).

[21]  Craig Gentry,et al.  A fully homomorphic encryption scheme , 2009 .

[22]  Yuval Ishai,et al.  Homomorphic Secret Sharing: Optimizations and Applications , 2017, CCS.

[23]  Lei Jiang,et al.  SHE: A Fast and Accurate Deep Neural Network for Encrypted Data , 2019, NeurIPS.

[24]  Elad Hoffer,et al.  Scalable Methods for 8-bit Training of Neural Networks , 2018, NeurIPS.

[25]  Raghuraman Krishnamoorthi,et al.  Quantizing deep convolutional networks for efficient inference: A whitepaper , 2018, ArXiv.

[26]  Marcel Keller,et al.  MP-SPDZ: A Versatile Framework for Multi-Party Computation , 2020, IACR Cryptol. ePrint Arch..

[27]  Marcus Peinado,et al.  High-Resolution Side Channels for Untrusted Operating Systems , 2017, USENIX Annual Technical Conference.

[28]  Vijay Arya,et al.  Model Extraction Warning in MLaaS Paradigm , 2017, ACSAC.

[29]  Varun Kanade,et al.  TAPAS: Tricks to Accelerate (encrypted) Prediction As a Service , 2018, ICML.

[30]  Craig Gentry,et al.  Fully homomorphic encryption using ideal lattices , 2009, STOC '09.

[31]  Shweta Shinde,et al.  Privado: Practical and Secure DNN Inference , 2018, ArXiv.

[32]  Nir Bitansky,et al.  Succinct Non-Interactive Arguments via Linear Interactive Proofs , 2013, Journal of Cryptology.

[33]  Dragos Rotaru,et al.  Maliciously Secure Matrix Multiplication with Applications to Private Deep Learning , 2020, IACR Cryptol. ePrint Arch..

[34]  Srdjan Capkun,et al.  Software Grand Exposure: SGX Cache Attacks Are Practical , 2017, WOOT.

[35]  Dan Boneh,et al.  Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware , 2018, ICLR.

[36]  Farinaz Koushanfar,et al.  XONN: XNOR-based Oblivious Deep Neural Network Inference , 2019, IACR Cryptol. ePrint Arch..

[37]  Oded Regev,et al.  On lattices, learning with errors, random linear codes, and cryptography , 2005, STOC '05.

[38]  Fan Zhang,et al.  Stealing Machine Learning Models via Prediction APIs , 2016, USENIX Security Symposium.

[39]  Carl A. Gunter,et al.  Leaky Cauldron on the Dark Land: Understanding Memory Side-Channel Hazards in SGX , 2017, CCS.

[40]  Farinaz Koushanfar,et al.  Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications , 2018, IACR Cryptol. ePrint Arch..

[41]  Abhi Shelat,et al.  The Cut-and-Choose Game and Its Application to Cryptographic Protocols , 2016, USENIX Security Symposium.

[42]  Farinaz Koushanfar,et al.  DeepSecure: Scalable Provably-Secure Deep Learning , 2017, 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC).

[43]  Kurt Keutzer,et al.  HAWQ: Hessian AWare Quantization of Neural Networks With Mixed-Precision , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[44]  Ran Gilad-Bachrach,et al.  Low Latency Privacy Preserving Inference , 2018, ICML.

[45]  Frederik Vercauteren,et al.  Somewhat Practical Fully Homomorphic Encryption , 2012, IACR Cryptol. ePrint Arch..

[46]  Aseem Rastogi,et al.  CrypTFlow2: Practical 2-Party Secure Inference , 2020, IACR Cryptol. ePrint Arch..

[47]  Jonathan Katz,et al.  Optimizing Authenticated Garbling for Faster Secure Two-Party Computation , 2018, IACR Cryptol. ePrint Arch..

[48]  Gorka Irazoqui Apecechea,et al.  CacheZoom: How SGX Amplifies The Power of Cache Attacks , 2017, CHES.

[49]  Andrew Chi-Chih Yao,et al.  How to Generate and Exchange Secrets (Extended Abstract) , 1986, FOCS.

[50]  Marcel Keller,et al.  Improved Primitives for MPC over Mixed Arithmetic-Binary Circuits , 2020, IACR Cryptol. ePrint Arch..

[51]  Sameer Wagh,et al.  SecureNN: 3-Party Secure Computation for Neural Network Training , 2019, Proc. Priv. Enhancing Technol..

[52]  Marcel Keller,et al.  Overdrive: Making SPDZ Great Again , 2018, IACR Cryptol. ePrint Arch..

[53]  Payman Mohassel,et al.  SecureML: A System for Scalable Privacy-Preserving Machine Learning , 2017, 2017 IEEE Symposium on Security and Privacy (SP).

[54]  Nishant Kumar,et al.  CrypTFlow: Secure TensorFlow Inference , 2020, 2020 IEEE Symposium on Security and Privacy (SP).

[55]  Raluca Ada Popa,et al.  Delphi: A Cryptographic Inference System for Neural Networks , 2020, IACR Cryptol. ePrint Arch..

[56]  Johannes Götzfried,et al.  Cache Attacks on Intel SGX , 2017, EUROSEC.

[57]  Yurong Liu,et al.  A survey of deep neural network architectures and their applications , 2017, Neurocomputing.

[58]  Michael O. Rabin,et al.  How To Exchange Secrets with Oblivious Transfer , 2005, IACR Cryptol. ePrint Arch..

[59]  Daniel J. Bernstein,et al.  Concrete quantum cryptanalysis of binary elliptic curves , 2020, IACR Cryptol. ePrint Arch..

[60]  Ivan Damgård,et al.  Multiparty Computation from Somewhat Homomorphic Encryption , 2012, IACR Cryptol. ePrint Arch..

[61]  Yao Lu,et al.  Oblivious Neural Network Predictions via MiniONN Transformations , 2017, IACR Cryptol. ePrint Arch..

[62]  Mihir Bellare,et al.  Foundations of garbled circuits , 2012, CCS.

[63]  Anca D. Dragan,et al.  Model Reconstruction from Model Explanations , 2018, FAT.

[64]  Anantha Chandrakasan,et al.  Gazelle: A Low Latency Framework for Secure Neural Network Inference , 2018, IACR Cryptol. ePrint Arch..

[65]  Ian Molloy,et al.  Defending Against Neural Network Model Stealing Attacks Using Deceptive Perturbations , 2019, 2019 IEEE Security and Privacy Workshops (SPW).

[66]  Eyal Kushilevitz,et al.  Falcon: Honest-Majority Maliciously Secure Framework for Private Deep Learning , 2021, Proc. Priv. Enhancing Technol..

[67]  Li Fei-Fei,et al.  Faster CryptoNets: Leveraging Sparsity for Real-World Encrypted Inference , 2018, ArXiv.

[68]  Thomas F. Wenisch,et al.  Foreshadow: Extracting the Keys to the Intel SGX Kingdom with Transient Out-of-Order Execution , 2018, USENIX Security Symposium.

[69]  Ilya Mironov,et al.  Cryptanalytic Extraction of Neural Network Models , 2020, CRYPTO.

[70]  Silvio Micali,et al.  The knowledge complexity of interactive proof-systems , 1985, STOC '85.