Towards Automatic Model Specialization for Edge Video Analytics

Judging by popular and generic computer vision challenges, such as the ILSVRC (ImageNet Large Scale Visual Recognition Challenge) or PASCAL VOC, neural networks have proven to be exceptionally accurate in some tasks, surpassing even humanlevel accuracy. However, state-of-the-art accuracy often comes at a high computational price, requiring equally state-of-the-art and high-end hardware acceleration to achieve anything near real-time performance. At the same time, use cases such as smart cities or autonomous vehicles require an automated analysis of images from fixed cameras in real-time. Due to the vast and constant network bandwidth these streams would generate, we cannot rely on offloading compute to the omnipresent and omnipotent cloud. Consequently, a distributed edge cloud should take over and process images locally. However, by nature, the edge cloud is resourceconstrained, which puts a limit on the computational complexity of the models executed in the edge. Nonetheless, there is a need for a meeting point between the edge cloud and accurate real-time video analytics. One solution is to use methods to specialize lightweight models on a per-camera basis but it quickly becomes unfeasible as the number of cameras grows unless the process is fully automated. In this paper, we present and evaluate COVA (Contextually Optimized Video Analytics), a framework to assist in the automatic specialization of models for video analytics in edge cloud cameras. COVA aims to automatically improve the accuracy of lightweight models through the automatic specialization of models. Moreover, we discuss and analyze each step involved in the process to understand the different trade-offs that each one entails. Additionally, we show how the sole assumption of static cameras allows us to make a series of considerations that greatly simplify the scope of the problem and, in turn, enables COVA to successfully specialize models using traditional computer vision techniques specifically chosen for the task. Through COVA, we show that complex neural networks, i.e., those able to generalize well, can be effectively used as teachers to annotate datasets for the specialization of lightweight neural networks and adapt them to the specific context in which they will be deployed. This allows us to tailor models to increase their accuracy while keeping their computational cost constant and do so without any human interaction. Results show that COVA can automatically improve pre-trained models by an average of 21% on the different scenes of the VIRAT dataset.