Federated Object Detection: Optimizing Object Detection Model with Federated Learning

Object detection with deep learning model has achieved good results in many fields, but in some fields that think highly of data privacy, such as medical care, its applications is greatly limited by data. And Federated Learning allows clients to train a model together, while leaving their data in the local, without sharing with the server or other clients. Using the methods of Federated Learning, such as Federated Averaging(FedAvg), to train models can provide privacy, security benefits. Nonetheless, there is little experiment applying Federated Learning algorithms to train the model with a large number of parameters, such as deep learning object detection model. With non-IID data, the accuracy of object detection model trained by FedAvg reduces significantly, and need more rounds to coverage. In this work, we use Kullback-Leibler divergence(KLD) measure the weights divergence between different model trained with non-IID data. And we propose a useful scheme to improve FedAvg based Abnormal Weights Supression, reducing the influence of the weights divergence caused by non-IID and unbalanced data. As a representative of object detection, we choose Single Shot MultiBox Detector(SSD) as the base model. The results of the experiments show that the Mean Average Precision(mAP) get obvious improvement in Pascal VOC 2007 test dataset.

[1]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.

[2]  Andre Esteva,et al.  A guide to deep learning in healthcare , 2019, Nature Medicine.

[3]  Yang Liu,et al.  Secure Federated Transfer Learning , 2018, ArXiv.

[4]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[5]  Vitaly Shmatikov,et al.  Privacy-preserving deep learning , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[6]  Xiang Li,et al.  On the Convergence of FedAvg on Non-IID Data , 2019, ICLR.

[7]  Yann LeCun,et al.  Deep learning with Elastic Averaging SGD , 2014, NIPS.

[8]  Sarvar Patel,et al.  Practical Secure Aggregation for Privacy-Preserving Machine Learning , 2017, IACR Cryptol. ePrint Arch..

[9]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[10]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[11]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[13]  Klaus-Robert Müller,et al.  Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[14]  William Shakespeare,et al.  Complete Works of William Shakespeare , 1854 .

[15]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[16]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  William J. Dally,et al.  Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training , 2017, ICLR.

[18]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[19]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[20]  Yue Zhao,et al.  Federated Learning with Non-IID Data , 2018, ArXiv.

[21]  Ming Liu,et al.  Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems , 2019, IEEE Robotics and Automation Letters.