VFL: A Verifiable Federated Learning With Privacy-Preserving for Big Data in Industrial IoT

Due to the strong analytical ability of big data, deep learning has been widely applied to train the collected data in industrial IoT. However, for privacy issues, traditional data-gathering centralized learning is not applicable to industrial scenarios sensitive to training sets. Recently, federated learning has received widespread attention, since it trains a model by only relying on gradient aggregation without accessing training sets. But existing researches reveal that the shared gradient still retains the sensitive information of the training set. Even worse, a malicious aggregation server may return forged aggregated gradients. In this paper, we propose the VFL, verifiable federated learning with privacy-preserving for big data in industrial IoT. Specifically, we use Lagrange interpolation to elaborately set interpolation points for verifying the correctness of the aggregated gradients. Compared with existing schemes, the verification overhead of VFL remains constant regardless of the number of participants. Moreover, we employ the blinding technology to protect the privacy of the gradients submitted by the participants. If no more than n-2 of n participants collude with the aggregation server, VFL could guarantee the encrypted gradients of other participants not being inverted. Experimental evaluations corroborate the practical performance of the presented VFL framework with high accuracy and efficiency.

[1]  Tianjian Chen,et al.  Federated Machine Learning: Concept and Applications , 2019 .

[2]  Kan Yang,et al.  VerifyNet: Secure and Verifiable Federated Learning , 2020, IEEE Transactions on Information Forensics and Security.

[3]  Huaqun Wang,et al.  Privacy-Preserving Federated Learning in Fog Computing , 2020, IEEE Internet of Things Journal.

[4]  Michael Naehrig,et al.  CryptoNets: applying neural networks to encrypted data with high throughput and accuracy , 2016, ICML 2016.

[5]  Jian Shen,et al.  Privacy preserving multi-party computation delegation for deep learning in cloud computing , 2018, Inf. Sci..

[6]  Naixue Xiong,et al.  Adaptive Range-Based Target Localization Using Diffusion Gauss–Newton Method in Industrial Environments , 2019, IEEE Transactions on Industrial Informatics.

[7]  Naixue Xiong,et al.  Intelligent Impulsive Synchronization of Nonlinear Interconnected Neural Networks for Image Protection , 2018, IEEE Transactions on Industrial Informatics.

[8]  Naixue Xiong,et al.  A Privacy-Preserving Outsourcing Scheme for Image Local Binary Pattern in Secure Industrial Internet of Things , 2020, IEEE Transactions on Industrial Informatics.

[9]  Anantha Chandrakasan,et al.  Gazelle: A Low Latency Framework for Secure Neural Network Inference , 2018, IACR Cryptol. ePrint Arch..

[10]  Anmin Fu,et al.  PRTA: A Proxy Re-encryption based Trusted Authorization scheme for nodes on CloudIoT , 2020, Inf. Sci..

[11]  Peter Richtárik,et al.  Federated Optimization: Distributed Machine Learning for On-Device Intelligence , 2016, ArXiv.

[12]  Shiho Moriai,et al.  Privacy-Preserving Deep Learning via Additively Homomorphic Encryption , 2019, 2019 IEEE 26th Symposium on Computer Arithmetic (ARITH).

[13]  David Mazières,et al.  On-the-fly verification of rateless erasure codes for efficient content distribution , 2004, IEEE Symposium on Security and Privacy, 2004. Proceedings. 2004.

[14]  Payman Mohassel,et al.  SecureML: A System for Scalable Privacy-Preserving Machine Learning , 2017, 2017 IEEE Symposium on Security and Privacy (SP).

[15]  Ivan Damgård,et al.  Multiparty Computation from Somewhat Homomorphic Encryption , 2012, IACR Cryptol. ePrint Arch..

[16]  Ion Stoica,et al.  Helen: Maliciously Secure Coopetitive Learning for Linear Models , 2019, 2019 IEEE Symposium on Security and Privacy (SP).

[17]  Emmanuel Bresson,et al.  A Simple Public-Key Cryptosystem with a Double Trapdoor Decryption Mechanism and Its Applications , 2003, ASIACRYPT.

[18]  Kazem Sohraby,et al.  IoT Considerations, Requirements, and Architectures for Smart Buildings—Energy Optimization and Next-Generation Building Management Systems , 2017, IEEE Internet of Things Journal.

[19]  Ju Ren,et al.  GANobfuscator: Mitigating Information Leakage Under GAN via Differential Privacy , 2019, IEEE Transactions on Information Forensics and Security.

[20]  Adi Shamir,et al.  How to share a secret , 1979, CACM.

[21]  George Mastorakis,et al.  Adaptive Positioning Systems Based on Multiple Wireless Interfaces for Industrial IoT in Harsh Manufacturing Environments , 2020, IEEE Journal on Selected Areas in Communications.

[22]  Amir Salman Avestimehr,et al.  Lagrange Coded Computing: Optimal Design for Resiliency, Security and Privacy , 2018, AISTATS.

[23]  Matt J. Kusner,et al.  QUOTIENT: Two-Party Secure Neural Network Training and Prediction , 2019, CCS.

[24]  Naixue Xiong,et al.  A Greedy Deep Learning Method for Medical Disease Analysis , 2018, IEEE Access.

[25]  Shui Yu,et al.  Security and Privacy in the Age of the Smart Internet of Things: An Overview from a Networking Perspective , 2018, IEEE Communications Magazine.

[26]  Guomin Yang,et al.  ESDRA: An Efficient and Secure Distributed Remote Attestation Scheme for IoT Swarms , 2019, IEEE Internet of Things Journal.

[27]  Pascal Paillier,et al.  Public-Key Cryptosystems Based on Composite Degree Residuosity Classes , 1999, EUROCRYPT.

[28]  Michael Zohner,et al.  ABY - A Framework for Efficient Mixed-Protocol Secure Two-Party Computation , 2015, NDSS.

[29]  Blaise Agüera y Arcas,et al.  Federated Learning of Deep Networks using Model Averaging , 2016, ArXiv.

[30]  Michael Naehrig,et al.  Improved Security for a Ring-Based Fully Homomorphic Encryption Scheme , 2013, IMACC.

[31]  Tong Li,et al.  NPMML: A Framework for Non-Interactive Privacy-Preserving Multi-Party Machine Learning , 2020, IEEE Transactions on Dependable and Secure Computing.

[32]  Huaqun Wang,et al.  A Privacy-Preserving and Verifiable Federated Learning Scheme , 2020, ICC 2020 - 2020 IEEE International Conference on Communications (ICC).

[33]  Siu-Ming Yiu,et al.  Multi-key privacy-preserving deep learning in cloud computing , 2017, Future Gener. Comput. Syst..

[34]  Amir Salman Avestimehr,et al.  CodedPrivateML: A Fast and Privacy-Preserving Framework for Distributed Machine Learning , 2019, IEEE Journal on Selected Areas in Information Theory.

[35]  Vitaly Shmatikov,et al.  Privacy-preserving deep learning , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[36]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[37]  Vinod Vaikuntanathan,et al.  On-the-fly multiparty computation on the cloud via multikey fully homomorphic encryption , 2012, STOC '12.

[38]  Moni Naor,et al.  Efficient oblivious transfer protocols , 2001, SODA '01.

[39]  Anfeng Liu,et al.  An intelligent incentive mechanism for coverage of data collection in cognitive internet of things , 2019, Future Gener. Comput. Syst..

[40]  Hovav Shacham,et al.  Aggregate and Verifiably Encrypted Signatures from Bilinear Maps , 2003, EUROCRYPT.

[41]  Robert H. Deng,et al.  Data Security Issues in Deep Learning: Attacks, Countermeasures, and Opportunities , 2019, IEEE Communications Magazine.

[42]  Naixue Xiong,et al.  A Fuzzy Probability Bayesian Network Approach for Dynamic Cybersecurity Risk Assessment in Industrial Control Systems , 2018, IEEE Transactions on Industrial Informatics.

[43]  Yang Song,et al.  Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning , 2018, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[44]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.