Live Video Analytics with FPGA-based Smart Cameras

Analyzing video feeds from large camera networks requires enormous compute and bandwidth. Edge computing has been proposed to ease the burden by bringing resources to the proximity of data. However, the number of cameras keeps growing and the associated computing resources on edge will again fall in short. To fundamentally solve the resource scarcity problem and make edge-based live video analytics scalable, we present an FPGA-based smart camera design that enables efficient in-situ streaming processing to meet the stringent low-power, energy-efficient, low-latency requirements of edge vision applications. By leveraging FPGA's intrinsic properties of architecture efficiency and exploiting its hardware support for parallelism, we demonstrate a 49x speedup over CPU and 6.4x more energy-efficiency than GPU, verified using a background subtraction algorithm.

[1]  Matei Zaharia,et al.  NoScope: Optimizing Deep CNN-Based Queries over Video Streams at Scale , 2017, Proc. VLDB Endow..

[2]  Shuxiang Guo,et al.  A low-power SoC-based moving target detection system for amphibious spherical robots , 2015, 2015 IEEE International Conference on Mechatronics and Automation (ICMA).

[3]  Paramvir Bahl,et al.  VideoEdge: Processing Camera Streams using Hierarchical Clusters , 2018, 2018 IEEE/ACM Symposium on Edge Computing (SEC).

[4]  Samvit Jain,et al.  Scaling Video Analytics Systems to Large Camera Deployments , 2018, HotMobile.

[5]  Paramvir Bahl,et al.  Live Video Analytics at Scale with Approximation and Delay-Tolerance , 2017, NSDI.

[6]  Nicholas D. Lane,et al.  DeepEye: Resource Efficient Local Execution of Multiple Deep Vision Models using Wearable Commodity Hardware , 2017, MobiSys.

[7]  Pong P. Chu.  FPGA Prototyping by VHDL Examples: Xilinx Spartan-3 Version , 2008 .

[8]  Paramvir Bahl,et al.  Video Analytics - Killer App for Edge Computing , 2019, MobiSys.

[9]  Russell Tessier,et al.  FPGA Architecture: Survey and Challenges , 2008, Found. Trends Electron. Des. Autom..

[10]  Ion Stoica,et al.  Chameleon: scalable adaptation of video analytics , 2018, SIGCOMM.

[11]  Paramvir Bahl,et al.  Glimpse: Continuous, Real-Time Object Recognition on Mobile Devices , 2015, SenSys.

[12]  Sek M. Chai,et al.  FPGA acceleration for feature based processing applications , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[13]  Oihana Otaegui,et al.  A reconfigurable embedded vision system for advanced driver assistance , 2015, Journal of Real-Time Image Processing.

[14]  Gregory R. Ganger,et al.  Mainstream: Dynamic Stem-Sharing for Multi-Tenant Video Processing , 2018, USENIX Annual Technical Conference.

[15]  Seungyeop Han,et al.  Fast Video Classification via Adaptive Cascading of Deep Models , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Alec Wolman,et al.  MCDNN: An Approximation-Based Execution Framework for Deep Stream Processing Under Resource Constraints , 2016, MobiSys.

[17]  Scott Fischaber,et al.  OpenCV based road sign recognition on Zynq , 2013, 2013 11th IEEE International Conference on Industrial Informatics (INDIN).

[18]  Xiao Zeng,et al.  NestDNN: Resource-Aware Multi-Tenant On-Device Deep Learning for Continuous Mobile Vision , 2018, MobiCom.

[19]  Yujie Zhang,et al.  Design of Face Detection System Based on FPGA , 2013, ICSI.

[20]  Z. Zivkovic.  Improved adaptive Gaussian mixture model for background subtraction , 2004, ICPR 2004.

[21]  Jason Cong,et al.  Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks , 2015, FPGA.

[22]  Rafal Kapela,et al.  Embedded platform for local image descriptor based object detection , 2015, Appl. Math. Comput..

[23]  Bo Wang,et al.  The face detection and location system based on Zynq , 2014, 2014 11th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD).

[24]  Hui Liu,et al.  On-Demand Deep Model Compression for Mobile Devices: A Usage-Driven Model Selection Framework , 2018, MobiSys.

[25]  Aakanksha Chowdhery,et al.  The Design and Implementation of a Wireless Video Surveillance System , 2015, MobiCom.

[26]  James Karp,et al.  Neutron, 64 MeV Proton, Thermal Neutron and Alpha Single-Event Upset Characterization of Xilinx 20nm UltraScale Kintex FPGA , 2015, 2015 IEEE Radiation Effects Data Workshop (REDW).

[27]  Lukás Sekanina,et al.  Image filter evolution on the Xilinx Zynq Platform , 2013, 2013 NASA/ESA Conference on Adaptive Hardware and Systems (AHS-2013).

[28]  Lin Zhong,et al.  Starfish: Efficient Concurrency Support for Computer Vision Applications , 2015, MobiSys.