A video cloud platform combing online and offline cloud computing technologies

Abstract With the rapid growth of video data from various sources, like security and transportation surveillance, there arise requirements for both online real-time analysis and offline batch processing of large-scale video data. Existing video processing systems fall short in addressing many challenges in large-scale video processing, for example performance, data storage, and fault tolerance. The emerging cloud computing and big data techniques shed lights to intelligent processing for large-scale video data. This paper proposes a general cloud-based architecture and platform that can provide a robust solution to intelligent analysis and storage for video data, which is named as BiF (Batch processing Integrated with Fast processing) architecture. We have implemented the BiF architecture using both Hadoop platform and Storm platform, which are typical offline batch processing cloud platform and online real-time processing cloud platform, respectively. The proposed architecture can handle continual surveillance video data effectively, where real-time analysis, batch processing, distributed storage and cloud services are seamlessly integrated to meet the requirements of video data processing and management. The evaluations show that the proposed approach is efficient in terms of performance, storage, and fault tolerance.

[1]  Myoungjin Kim,et al.  Towards Efficient Design and Implementation of a Hadoop-based Distributed Video Transcoding System in Cloud Computing Environment , 2013 .

[2]  Weishan Zhang,et al.  A hybrid approach to self-management in a pervasive service middleware , 2014, Knowl. Based Syst..

[3]  Sanjaya Shankar Tripathy,et al.  Detection of human presence in a surveillance video using fuzzy approach , 2014, 2014 International Conference on Signal Processing and Integrated Networks (SPIN).

[4]  M. N. Vora,et al.  Hadoop-HBase for large-scale data , 2011, Proceedings of 2011 International Conference on Computer Science and Network Technology.

[5]  Weishan Zhang,et al.  Online Multiperson Tracking and Counting with Cloud Computing , 2014, 2014 International Conference on Identification, Information and Knowledge in the Internet of Things.

[6]  S. M. Rahman,et al.  Video surveillance in the cloud-computing? , 2012, 2012 7th International Conference on Electrical and Computer Engineering.

[7]  Weishan Zhang,et al.  A Realtime Framework for Video Object Detection with Storm , 2014, 2014 IEEE 11th Intl Conf on Ubiquitous Intelligence and Computing and 2014 IEEE 11th Intl Conf on Autonomic and Trusted Computing and 2014 IEEE 14th Intl Conf on Scalable Computing and Communications and Its Associated Workshops.

[8]  Paul Clements,et al.  Software architecture in practice , 1999, SEI series in software engineering.

[9]  Kimiaki Shirahama.  Intelligent video processing using data mining techniques , 2011, ACMMR.

[10]  Zezhi Chen,et al.  A self-adaptive Gaussian mixture model , 2014, Comput. Vis. Image Underst..

[11]  Euiseong Seo,et al.  Extensible Video Processing Framework in Apache Hadoop , 2013, 2013 IEEE 5th International Conference on Cloud Computing Technology and Science.

[12]  Aamir Saeed Malik,et al.  Proposed framework of Intelligent Video Automatic Target Recognition System (IVATRs) , 2011, 2011 National Postgraduate Conference.

[13]  Ian D. Reid,et al.  Stable multi-target tracking in real-time surveillance video , 2011, CVPR 2011.

[14]  Michael Miller,et al.  Cloud Computing: Web-Based Applications That Change the Way You Work and Collaborate Online , 2008 .

[15]  Huiyu Zhou,et al.  Feature extraction and clustering for dynamic video summarisation , 2010, Neurocomputing.

[16]  Touradj Ebrahimi,et al.  Tracking video objects in cluttered background , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[18]  Tom White,et al.  Hadoop: The Definitive Guide , 2009 .

[19]  Xue-wen Chen,et al.  Large-Scale Deep Belief Nets With MapReduce , 2014, IEEE Access.

[20]  Nathan Marz,et al.  Big Data: Principles and best practices of scalable realtime data systems , 2015 .

[21]  Xindong Wu,et al.  Video data mining: semantic indexing and event detection from the association perspective , 2005, IEEE Transactions on Knowledge and Data Engineering.

[22]  Jung-Hwan Oh,et al.  Multimedia Data Mining Framework for Raw Video Sequences , 2002, Revised Papers from MDM/KDD and PAKDD/KDMCD.

[23]  GhemawatSanjay,et al.  The Google file system , 2003 .

[24]  Kimiaki Shirahama,et al.  Video data mining: rhythms in a movie , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[25]  Roy T. Fielding,et al.  Principled design of the modern Web architecture , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[26]  Asadullah Shah,et al.  Critical Insight for MapReduce Optimization in Hadoop , 2014 .