SIAT: A Distributed Video Analytics Framework for Intelligent Video Surveillance

In recent years, the amount of intelligent CCTV cameras installed in public places for surveillance has increased enormously and as a result, a large amount of video data is produced every moment. Due to this situation, there is an increasing request for the distributed processing of large-scale video data. In an intelligent video analytics platform, a submitted unstructured video undergoes through several multidisciplinary algorithms with the aim of extracting insights and making them searchable and understandable for both human and machine. Video analytics have applications ranging from surveillance to video content management. In this context, various industrial and scholarly solutions exist. However, most of the existing solutions rely on a traditional client/server framework to perform face and object recognition while lacking the support for more complex application scenarios. Furthermore, these frameworks are rarely handled in a scalable manner using distributed computing. Besides, existing works do not provide any support for low-level distributed video processing APIs (Application Programming Interfaces). They also failed to address a complete service-oriented ecosystem to meet the growing demands of consumers, researchers and developers. In order to overcome these issues, in this paper, we propose a distributed video analytics framework for intelligent video surveillance known as SIAT. The proposed framework is able to process both the real-time video streams and batch video analytics. Each real-time stream also corresponds to batch processing data. Hence, this work correlates with the symmetry concept. Furthermore, we introduce a distributed video processing library on top of Spark. SIAT exploits state-of-the-art distributed computing technologies with the aim to ensure scalability, effectiveness and fault-tolerance. Lastly, we implant and evaluate our proposed framework with the goal to authenticate our claims.

[1]  M. Anwar Hossain,et al.  Framework for a Cloud-Based Multimedia Surveillance System , 2014, Int. J. Distributed Sens. Networks.

[2]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Mohammed Ghazal,et al.  A Modular Distributed Video Surveillance System Over IP , 2006, 2006 Canadian Conference on Electrical and Computer Engineering.

[4]  Lars George,et al.  HBase - The Definitive Guide: Random Access to Your Planet-Size Data , 2011 .

[5]  Jay Kreps,et al.  Kafka : a Distributed Messaging System for Log Processing , 2011 .

[6]  Qi Huang,et al.  SVE: Distributed Video Processing at Facebook Scale , 2017, SOSP.

[7]  Wu Chao,et al.  Multi-agent Based Distributed Video Surveillance System over IP , 2008, 2008 International Symposium on Computer Science and Computational Technology.

[8]  Masato Oguchi,et al.  A study of a video analysis framework using Kafka and spark streaming , 2017, 2017 IEEE International Conference on Big Data (Big Data).

[9]  Yao Zhang,et al.  BigDL: A Distributed Deep Learning Framework for Big Data , 2019, SoCC.

[10]  Yang Wang,et al.  BigDL: A Distributed Deep Learning Framework for Big Data , 2018, SoCC.

[11]  Ameet Talwalkar,et al.  MLlib: Machine Learning in Apache Spark , 2015, J. Mach. Learn. Res..

[12]  Sung Wook Baik,et al.  Adaptive key frame extraction for video summarization using an aggregation mechanism , 2012, J. Vis. Commun. Image Represent..

[13]  Andrew Zisserman,et al.  Two-Stream Convolutional Networks for Action Recognition in Videos , 2014, NIPS.

[14]  Chang Liu,et al.  PF-Face: A Parallel Framework for Face Classification and Search from Massive Videos Based on Spark , 2018, 2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM).

[15]  Young-Koo Lee,et al.  Human Action Recognition Using Adaptive Local Motion Descriptor in Spark , 2017, IEEE Access.

[16]  Scott Shenker,et al.  Discretized streams: fault-tolerant streaming computation at scale , 2013, SOSP.

[17]  Haitao Zhang,et al.  Efficient Online Surveillance Video Processing Based on Spark Framework , 2016, BigCom.

[18]  Melike Sah,et al.  Semantic annotation of surveillance videos for abnormal crowd behaviour search and analysis , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[19]  Barbara Caputo,et al.  Recognizing human actions: a local SVM approach , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[20]  Bo Xiao,et al.  Large-scale human action recognition with spark , 2015, 2015 IEEE 17th International Workshop on Multimedia Signal Processing (MMSP).

[21]  Wei-Han Chang,et al.  A fast MPEG-7 dominant color extraction with new similarity measure for image retrieval , 2008, J. Vis. Commun. Image Represent..

[22]  Richard Hill,et al.  Cloud-based scalable object detection and classification in video streams , 2018, Future Gener. Comput. Syst..

[23]  Hairong Kuang,et al.  The Hadoop Distributed File System , 2010, 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST).

[24]  Young-Koo Lee,et al.  Dynamic Scene Recognition Using Spatiotemporal Based DLTP on Spark , 2018, IEEE Access.

[25]  Weijia Xu,et al.  Enabling versatile analysis of large scale traffic video data with deep learning and HiveQL , 2017, 2017 IEEE International Conference on Big Data (Big Data).

[26]  Iraj Sodagar,et al.  Demonstration of the MPEG-2, MPEG-4 and H.263 video coding standards , 1997, Proceedings of First Signal Processing Society Workshop on Multimedia Signal Processing.

[27]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[28]  Shuai Yang,et al.  Efficient large scale near-duplicate video detection base on spark , 2016, 2016 IEEE International Conference on Big Data (Big Data).

[29]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Jian Wang,et al.  Building an intelligent video and image analysis evaluation platform for public security , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[31]  William Robson Schwartz,et al.  A scalable and flexible framework for smart video surveillance , 2016, Comput. Vis. Image Underst..

[32]  Mubarak Shah,et al.  Recognizing 50 human action categories of web videos , 2012, Machine Vision and Applications.

[33]  R. Maini Study and Comparison of Various Image Edge Detection Techniques , 2004 .

[34]  Michael J. Franklin,et al.  Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing , 2012, NSDI.

[35]  Weishan Zhang,et al.  A video cloud platform combing online and offline cloud computing technologies , 2015, Personal and Ubiquitous Computing.

[36]  Eduardo de Leon,et al.  Flexible and Scalable Deep Learning with MMLSpark , 2018, PAPIs.

[37]  A. Mahmood,et al.  Canny edge detection and Hough transform for high resolution video streams using Hadoop and Spark , 2019, Cluster Computing.