BLAZE: Blazing Fast Privacy-Preserving Machine Learning

Machine learning tools have illustrated their potential in many significant sectors such as healthcare and finance, to aide in deriving useful inferences. The sensitive and confidential nature of the data, in such sectors, raise natural concerns for the privacy of data. This motivated the area of Privacy-preserving Machine Learning (PPML) where privacy of the data is guaranteed. Typically, ML techniques require large computing power, which leads clients with limited infrastructure to rely on the method of Secure Outsourced Computation (SOC). In SOC setting, the computation is outsourced to a set of specialized and powerful cloud servers and the service is availed on a pay-per-use basis. In this work, we explore PPML techniques in the SOC setting for widely used ML algorithms-- Linear Regression, Logistic Regression, and Neural Networks. We propose BLAZE, a blazing fast PPML framework in the three server setting tolerating one malicious corruption over a ring (\Z{\ell}). BLAZE achieves the stronger security guarantee of fairness (all honest servers get the output whenever the corrupt server obtains the same). Leveraging an input-independent preprocessing phase, BLAZE has a fast input-dependent online phase relying on efficient PPML primitives such as: (i) A dot product protocol for which the communication in the online phase is independent of the vector size, the first of its kind in the three server setting; (ii) A method for truncation that shuns evaluating expensive circuit for Ripple Carry Adders (RCA) and achieves a constant round complexity. This improves over the truncation method of ABY3 (Mohassel et al., CCS 2018) that uses RCA and consumes a round complexity that is of the order of the depth of RCA. An extensive benchmarking of BLAZE for the aforementioned ML algorithms over a 64-bit ring in both WAN and LAN settings shows massive improvements over ABY3.

[1]  Jaideep Vaidya,et al.  Privacy-Preserving SVM Classification on Vertically Partitioned Data , 2006, PAKDD.

[2]  Marcel Keller,et al.  Overdrive: Making SPDZ Great Again , 2018, IACR Cryptol. ePrint Arch..

[3]  Dan Bogdanov,et al.  Students and Taxes: a Privacy-Preserving Social Study Using Secure Computation , 2015, IACR Cryptol. ePrint Arch..

[4]  Rafail Ostrovsky,et al.  Secure two-party k-means clustering , 2007, CCS '07.

[5]  Wenliang Du,et al.  Privacy-preserving cooperative scientific computations , 2001, Proceedings. 14th IEEE Computer Security Foundations Workshop, 2001..

[6]  Yuval Ishai,et al.  Efficient Multi-party Computation over Rings , 2003, EUROCRYPT.

[7]  Payman Mohassel,et al.  SecureML: A System for Scalable Privacy-Preserving Machine Learning , 2017, 2017 IEEE Symposium on Security and Privacy (SP).

[8]  Marcel Keller,et al.  Practical Covertly Secure MPC for Dishonest Majority - Or: Breaking the SPDZ Limits , 2013, ESORICS.

[9]  Yehuda Lindell,et al.  Fast Large-Scale Honest-Majority MPC for Malicious Adversaries , 2018, Journal of Cryptology.

[10]  Michael Zohner,et al.  ABY - A Framework for Efficient Mixed-Protocol Secure Two-Party Computation , 2015, NDSS.

[11]  Rafail Ostrovsky,et al.  Near-Linear Unconditionally-Secure Multiparty Computation with a Dishonest Minority , 2012, CRYPTO.

[12]  Ashish Choudhury,et al.  An Efficient Framework for Unconditionally Secure Multiparty Computation , 2017, IEEE Transactions on Information Theory.

[13]  Dan Bogdanov,et al.  Students and Taxes: a Privacy-Preserving Study Using Secure Computation , 2016, Proc. Priv. Enhancing Technol..

[14]  Arun Joseph,et al.  Fast Secure Computation for Small Population over the Internet , 2018, IACR Cryptol. ePrint Arch..

[15]  Aysegul Gunduz,et al.  A comparative analysis of speech signal processing algorithms for Parkinson's disease classification and the use of the tunable Q-factor wavelet transform , 2019, Appl. Soft Comput..

[16]  Yehuda Lindell,et al.  Privacy Preserving Data Mining , 2002, Journal of Cryptology.

[17]  Yehuda Lindell,et al.  High-Throughput Semi-Honest Secure Three-Party Computation with an Honest Majority , 2016, IACR Cryptol. ePrint Arch..

[18]  Martin Hirt,et al.  Perfectly-Secure MPC with Linear Communication Complexity , 2008, TCC.

[19]  Ivan Damgård,et al.  Yet Another Compiler for Active Security or: Efficient MPC Over Arbitrary Rings , 2018, IACR Cryptol. ePrint Arch..

[20]  Ramy Arnaout,et al.  Fast and accurate classification of echocardiograms using deep learning , 2017, ArXiv.

[21]  Farinaz Koushanfar,et al.  Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications , 2018, IACR Cryptol. ePrint Arch..

[22]  Yehuda Lindell,et al.  A Framework for Constructing Fast MPC over Arithmetic Circuits with Malicious Adversaries and an Honest-Majority , 2017, IACR Cryptol. ePrint Arch..

[23]  Frederik Vercauteren,et al.  EPIC: Efficient Private Image Classification (or: Learning from the Masters) , 2019, CT-RSA.

[24]  Donald Beaver,et al.  Efficient Multiparty Protocols Using Circuit Randomization , 1991, CRYPTO.

[25]  Marcel Keller,et al.  An architecture for practical actively secure MPC with dishonest majority , 2013, IACR Cryptol. ePrint Arch..

[26]  Peter Rindal,et al.  ABY3: A Mixed Protocol Framework for Machine Learning , 2018, IACR Cryptol. ePrint Arch..

[27]  Yehuda Lindell,et al.  High-Throughput Secure Three-Party Computation for Malicious Adversaries and an Honest Majority , 2017, IACR Cryptol. ePrint Arch..

[28]  Marcel Keller,et al.  MASCOT: Faster Malicious Arithmetic Secure Computation with Oblivious Transfer , 2016, IACR Cryptol. ePrint Arch..

[29]  Ashish Choudhury,et al.  ASTRA: High Throughput 3PC over Rings with Application to Secure Prediction , 2019, IACR Cryptol. ePrint Arch..

[30]  Marcel Keller,et al.  New Primitives for Actively-Secure MPC over Rings with Applications to Private Machine Learning , 2019, 2019 IEEE Symposium on Security and Privacy (SP).

[31]  Anat Paskin-Cherniavsky,et al.  Secure Computation with Minimal Interaction, Revisited , 2015, CRYPTO.

[32]  Donald Beaver,et al.  Precomputing Oblivious Transfer , 1995, CRYPTO.

[33]  John Launchbury,et al.  Application-Scale Secure Multiparty Computation , 2014, ESOP.

[34]  Ivan Damgård,et al.  Multiparty Computation from Somewhat Homomorphic Encryption , 2012, IACR Cryptol. ePrint Arch..

[35]  Yehuda Lindell,et al.  Optimized Honest-Majority MPC for Malicious Adversaries — Breaking the 1 Billion-Gate Per Second Barrier , 2017, 2017 IEEE Symposium on Security and Privacy (SP).

[36]  Xiaodong Lin,et al.  Privacy preserving regression modelling via distributed computation , 2004, KDD.

[37]  Ye Zhang,et al.  Fast and Secure Three-party Computation: The Garbled Circuit Approach , 2015, IACR Cryptol. ePrint Arch..

[38]  Martin Hirt,et al.  Efficient Multi-party Computation with Dispute Control , 2006, TCC.

[39]  Aleksandra B. Slavkovic,et al.  "Secure" Logistic Regression of Horizontally and Vertically Partitioned Distributed Databases , 2007, Seventh IEEE International Conference on Data Mining Workshops (ICDMW 2007).

[40]  Peter Sebastian Nordholt,et al.  Minimising Communication in Honest-Majority MPC by Batchwise Multiplication Verification , 2018, IACR Cryptol. ePrint Arch..

[41]  Xavier Bresson,et al.  FMA: A Dataset for Music Analysis , 2016, ISMIR.

[42]  Aleksandra Slavkovic,et al.  "Secure" Logistic Regression of Horizontally and Vertically Partitioned Distributed Databases , 2007 .

[43]  Ivan Damgård,et al.  Better Preprocessing for Secure Multiparty Computation , 2016, ACNS.

[44]  Rebecca N. Wright,et al.  Privacy-preserving distributed k-means clustering over arbitrarily partitioned data , 2005, KDD '05.

[45]  Dan Bogdanov,et al.  Deploying Secure Multi-Party Computation for Financial Data Analysis - (Short Paper) , 2012, Financial Cryptography.

[46]  Sameer Wagh,et al.  SecureNN: 3-Party Secure Computation for Neural Network Training , 2019, Proc. Priv. Enhancing Technol..

[47]  Yuval Ishai,et al.  Zero-Knowledge Proofs on Secret-Shared Data via Fully Linear PCPs , 2019, CRYPTO.

[48]  K. Hamidieh A data-driven statistical model for predicting the critical temperature of a superconductor , 2018, Computational Materials Science.

[49]  Arpita Patra,et al.  On the Exact Round Complexity of Secure Three-Party Computation , 2018, Journal of Cryptology.

[50]  Daniel E. Escudero,et al.  SPDℤ 2 k : Efficient MPC mod 2 k for Dishonest Majority. , 2018 .

[51]  Jaideep Vaidya,et al.  Knowledge and Information Systems , 2007 .

[52]  Mohammad Anagreh,et al.  Yet Another Compiler for Active Security or : Efficient MPC Over Arbitrary Rings , 2017 .

[53]  Ramy Arnaout,et al.  Fast and accurate view classification of echocardiograms using deep learning , 2018, npj Digital Medicine.

[54]  Dan Bogdanov,et al.  Sharemind: A Framework for Fast Privacy-Preserving Computations , 2008, ESORICS.