EdgeMA: Model Adaptation System for Real-Time Video Analytics on Edge Devices

Real-time video analytics on edge devices for changing scenes remains a difficult task. As edge devices are usually resource-constrained, edge deep neural networks (DNNs) have fewer weights and shallower architectures than general DNNs. As a result, they only perform well in limited scenarios and are sensitive to data drift. In this paper, we introduce EdgeMA, a practical and efficient video analytics system designed to adapt models to shifts in real-world video streams over time, addressing the data drift problem. EdgeMA extracts the gray level co-occurrence matrix based statistical texture feature and uses the Random Forest classifier to detect the domain shift. Moreover, we have incorporated a method of model adaptation based on importance weighting, specifically designed to update models to cope with the label distribution shift. Through rigorous evaluation of EdgeMA on a real-world dataset, our results illustrate that EdgeMA significantly improves inference accuracy.

[1]  Xiaoyang Qu,et al.  FedET: A Communication-Efficient Federated Class-Incremental Learning Framework Based on Enhanced Transformer , 2023, IJCAI.

[2]  Liang Wang,et al.  Shoggoth: Towards Efficient Edge-Cloud Collaborative Real-Time Video Inference via Adaptive Online Learning , 2023, ArXiv.

[3]  Nanqing Dong,et al.  Edge Computing with Artificial Intelligence: A Machine Learning Perspective , 2022, ACM Comput. Surv..

[4]  Katarina Grolinger,et al.  Edge-Cloud Computing for Internet of Things Data Analytics: Embedding Intelligence in the Edge With Deep Learning , 2021, IEEE Transactions on Industrial Informatics.

[5]  Paramvir Bahl,et al.  Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers , 2020, NSDI.

[6]  Xiaoyang Qu,et al.  Quantization and Knowledge Distillation for Efficient Federated Learning on Edge Devices , 2020, 2020 IEEE 22nd International Conference on High Performance Computing and Communications; IEEE 18th International Conference on Smart City; IEEE 6th International Conference on Data Science and Systems (HPCC/SmartCity/DSS).

[7]  Hui Zhang,et al.  An Efficient Joint Training Framework for Robust Small-Footprint Keyword Spotting , 2020, ICONIP.

[8]  H. Bal,et al.  Clownfish: Edge and Cloud Symbiosis for Video Stream Analytics , 2020, 2020 IEEE/ACM Symposium on Edge Computing (SEC).

[9]  Mehrdad Khani Shirkoohi,et al.  Real-Time Video Inference on Edge Devices via Adaptive Model Streaming , 2020, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[10]  Masashi Sugiyama,et al.  Rethinking Importance Weighting for Deep Learning under Distribution Shift , 2020, NeurIPS.

[11]  Hong-Yuan Mark Liao,et al.  YOLOv4: Optimal Speed and Accuracy of Object Detection , 2020, ArXiv.

[12]  Mohamed R. Ibrahim,et al.  WeatherNet: Recognising weather and visual conditions from street-level images using deep residual learning , 2019, ISPRS Int. J. Geo Inf..

[13]  Yizhou Wang,et al.  Multi-Level Domain Adaptive Learning for Cross-Domain Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[14]  F. Richard Yu,et al.  Power-Constrained Edge Computing With Maximum Processing Capacity for IoT Networks , 2019, IEEE Internet of Things Journal.

[15]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[16]  Ion Stoica,et al.  Chameleon: scalable adaptation of video analytics , 2018, SIGCOMM.

[17]  Alexander J. Smola,et al.  Detecting and Correcting for Label Shift with Black Box Predictors , 2018, ICML.

[18]  Paramvir Bahl,et al.  Focus: Querying Large Video Datasets with Low Latency and Low Cost , 2018, OSDI.

[19]  Xiangyu Zhang,et al.  ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[21]  Paramvir Bahl,et al.  Live Video Analytics at Scale with Approximation and Delay-Tolerance , 2017, NSDI.

[22]  Weisong Shi,et al.  The Promise of Edge Computing , 2016, Computer.

[23]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Ming-Hsuan Yang,et al.  UA-DETRAC: A new benchmark and protocol for multi-object detection and tracking , 2015, Comput. Vis. Image Underst..

[25]  Stephen J. Wright.  Coordinate descent algorithms , 2015, Mathematical Programming.

[26]  Yurii Nesterov,et al.  Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems , 2012, SIAM J. Optim..

[27]  L. Breiman.  Random Forests , 2001, Encyclopedia of Machine Learning and Data Mining.

[28]  Kevin Hsieh,et al.  RECL: Responsive Resource-Efficient Continuous Learning for Video Analytics , 2023, NSDI.

[29]  Feng-Feng Wei,et al.  Combining Traffic Assignment and Traffic Signal Control for Online Traffic Flow Optimization , 2022, ICONIP.

[30]  Shradha Verma,et al.  RVFL Classifier Based Ensemble Deep Learning for Early Diagnosis of Alzheimer's Disease , 2022, ICONIP.

[31]  Zhigang Luo,et al.  Counterfactual Causal Adversarial Networks for Domain Adaptation , 2022, ICONIP.

[32]  Francisco Herrera,et al.  A unifying view on dataset shift in classification , 2012, Pattern Recognit..