A Selective Model Aggregation Approach in Federated Learning for Online Anomaly Detection

Cloud computing has established a convenient approach for computing offloading, where the data produced by edge devices is gathered and processed in a centralized server. However, it results in critical issues related to latency. Recently, a neural network-based on-device learning approach is proposed, which offers a solution to the latency problem by relocating processing data to edge devices; even so, a single edge device may face insufficient training data to train a high-quality model, because of its limited available processing capabilities and energy resources. To address this issue, we extend the work to a federated learning system which enables the edge devices to exchange their trained parameters and update local models. However, in federated learning for anomaly detection, the reliability of local models would be different. For example, a number of trained models are likely to contain the features of anomalous data because of noise corruption or anomaly detection failure. Besides, as the communication protocol amongst edges could be exploited by attackers, the training data or model weights may have potential risks of being poisoned. Therefore, when we design a federated training algorithm, we should carefully select the local models that participate in model aggregation. In this work, we leverage an observed dataset to compute prediction errors, so that the unsatisfying local models can be excluded from federated training. Experimental results show that the federated learning approach improves anomaly detection accuracy. Besides, the proposed model aggregation solution achieves obvious improvement compared with the popular Federated Averaging method.

[1]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[2]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[3]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[4]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[5]  Stratis Ioannidis,et al.  Privacy-Preserving Ridge Regression on Hundreds of Millions of Records , 2013, 2013 IEEE Symposium on Security and Privacy.

[6]  Ameet Talwalkar,et al.  Federated Multi-Task Learning , 2017, NIPS.

[7]  Jinoh Kim,et al.  A survey of deep learning-based network anomaly detection , 2017, Cluster Computing.

[8]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[9]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[10]  Yanjiao Chen,et al.  InPrivate Digging: Enabling Tree-based Distributed Data Mining with Differential Privacy , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[11]  Sanjay Chawla,et al.  Anomaly Detection using One-Class Neural Networks , 2018, ArXiv.

[12]  Sebastian Caldas,et al.  LEAF: A Benchmark for Federated Settings , 2018, ArXiv.

[13]  Yang Liu,et al.  Secure Federated Transfer Learning , 2018, ArXiv.

[14]  Abnormal Client Behavior Detection in Federated Learning , 2019, ArXiv.

[15]  Prateek Mittal,et al.  Analyzing Federated Learning through an Adversarial Lens , 2018, ICML.

[16]  Hubert Eichner,et al.  Towards Federated Learning at Scale: System Design , 2019, SysML.

[17]  Albert Y. Zomaya,et al.  A Hybrid Deep Learning-Based Model for Anomaly Detection in Cloud Datacenter Networks , 2019, IEEE Transactions on Network and Service Management.

[18]  Le Trieu Phong,et al.  Privacy-Preserving Deep Learning via Weight Transmission , 2018, IEEE Transactions on Information Forensics and Security.

[19]  Mohsen Guizani,et al.  Bringing Deep Learning at the Edge of Information-Centric Internet of Things , 2019, IEEE Communications Letters.

[20]  Bo Zong,et al.  A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data , 2018, AAAI.

[21]  Shenghuo Zhu,et al.  Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning , 2018, AAAI.

[22]  Takayuki Nishio,et al.  Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge , 2018, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[23]  Mehrdad Mahdavi,et al.  Adaptive Personalized Federated Learning , 2020, ArXiv.

[24]  An-Yeu Andy Wu,et al.  Online Extreme Learning Machine Design for the Application of Federated Learning , 2020, 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS).

[25]  Hiroki Matsutani,et al.  A Neural Network-Based On-Device Learning Anomaly Detector for Edge Devices , 2019, IEEE Transactions on Computers.

[26]  Bingsheng He,et al.  A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection , 2019, IEEE Transactions on Knowledge and Data Engineering.

[27]  Qiang Wang,et al.  Data Poisoning Attacks on Federated Machine Learning , 2020, IEEE Internet of Things Journal.