Event BAGGING: A novel event summarization approach in multiview surveillance videos

Due to advancement in digital multimedia technologies, huge amount of video contents in such arenas as sports, surveillance, news, etc., is becoming widely available. Due to factors like presence of inactive frames, inter-view dependencies and frequent illumination changes, whole video content may not be important. Moreover, users may not have the adequate time to access/ store the entire video in real time. We propose a machine learning ensemble method to summarize the events in multiview videos, as a solution to the aforementioned issues. In order to make the accurate decision on keyframes, we trained our ensembles using meta approach where the inter-dependency and illumination changes of views are considered in the training phase. Experimental results on two benchmark datasets show that our model outperforms the state-of-the-art models with the best Recall and F-measure. The computing analysis of the model also indicates that it meets all requirements of real-time applications.

[1]  Bruce A. Draper,et al.  Bagging in computer vision , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[2]  Yueting Zhuang,et al.  Adaptive key frame extraction using unsupervised clustering , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[3]  Gang Hua,et al.  Semantic Model Vectors for Complex Video Event Recognition , 2012, IEEE Transactions on Multimedia.

[4]  Meng Wang,et al.  Event Driven Web Video Summarization by Tag Localization and Key-Shot Identification , 2012, IEEE Transactions on Multimedia.

[5]  Jiebo Luo,et al.  Towards Scalable Summarization of Consumer Videos Via Sparse Dictionary Selection , 2012, IEEE Transactions on Multimedia.

[6]  Changsheng Xu,et al.  A Generic Framework for Video Annotation via Semi-Supervised Learning , 2012, IEEE Transactions on Multimedia.

[7]  Xirong Li,et al.  TagBook: A Semantic Video Representation Without Supervision for Event Detection , 2015, IEEE Transactions on Multimedia.

[8]  Zhi-Hua Zhou,et al.  Multi-View Video Summarization , 2010, IEEE Transactions on Multimedia.

[9]  Regunathan Radhakrishnan,et al.  Effective and efficient sports highlights extraction using the minimum description length criterion in selecting GMM structures , 2004, ICME.

[10]  R. K. Agrawal,et al.  Combination of Kullback–Leibler divergence and Manhattan distance measures to detect salient objects , 2015, Signal Image Video Process..

[11]  Lie Lu,et al.  A generic framework of user attention model and its application in video summarization , 2005, IEEE Trans. Multim..

[12]  R. K. Agrawal,et al.  A convex hull approach in conjunction with Gaussian mixture model for salient object detection , 2016, Digit. Signal Process..

[13]  Tao Mei,et al.  A Bag-of-Importance Model With Locality-Constrained Coding Based Feature Learning for Video Summarization , 2014, IEEE Transactions on Multimedia.

[14]  R. K. Agrawal,et al.  A novel approach to combine features for salient object detection using constrained particle swarm optimization , 2014, Pattern Recognit..

[15]  Tao Mei,et al.  Super Fast Event Recognition in Internet Videos , 2015, IEEE Transactions on Multimedia.

[16]  Yi Yang,et al.  Semi-Supervised Multiple Feature Analysis for Action Recognition , 2014, IEEE Transactions on Multimedia.

[17]  R. K. Agrawal,et al.  A novel hybrid approach for salient object detection using local and global saliency in frequency domain , 2015, Multimedia Tools and Applications.

[18]  M. Shamim Hossain,et al.  Automatic Visual Concept Learning for Social Event Understanding , 2015, IEEE Transactions on Multimedia.

[19]  Torsten Hothorn,et al.  Bundling Classifiers by Bagging Trees , 2002, Comput. Stat. Data Anal..

[20]  Yanwei Fu,et al.  Multi-view Metric Learning for Multi-view Video Summarization , 2014, 2016 International Conference on Cyberworlds (CW).

[21]  B. S. Manjunath,et al.  Multicamera video summarization and anomaly detection from activity motifs , 2014, TOSN.

[22]  Chia-han Lee,et al.  On-Line Multi-View Video Summarization for Wireless Video Sensor Network , 2015, IEEE Journal of Selected Topics in Signal Processing.

[23]  Masoud Mazloom,et al.  Conceptlets: Selective Semantics for Classifying Video Events , 2014, IEEE Transactions on Multimedia.

[24]  Xi Wang,et al.  Fast Summarization of User-Generated Videos: Exploiting Semantic, Emotional, and Quality Clues , 2016, IEEE MultiMedia.

[25]  R. K. Agrawal,et al.  A novel position prior using fusion of rule of thirds and image center for salient object detection , 2016, Multimedia Tools and Applications.

[26]  Deepti D. Shrimankar,et al.  Equal Partition Based Clustering Approach for Event Summarization in Videos , 2016, 2016 12th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS).

[27]  Chong-Wah Ngo,et al.  Video Event Detection Using Motion Relativity and Feature Selection , 2014, IEEE Transactions on Multimedia.

[28]  Ananda S. Chowdhury,et al.  Multi-View Video Summarization Using Bipartite Matching Constrained Optimum-Path Forest Clustering , 2015, IEEE Transactions on Multimedia.

[29]  Krishan Kumar,et al.  Economically Efficient Virtualization over Cloud Using Docker Containers , 2016, 2016 IEEE International Conference on Cloud Computing in Emerging Markets (CCEM).

[30]  Changsheng Xu,et al.  Multi-Modal Event Topic Model for Social Event Analysis , 2016, IEEE Transactions on Multimedia.

[31]  Chong-Wah Ngo,et al.  Summarizing Rushes Videos by Motion, Object, and Event Understanding , 2012, IEEE Transactions on Multimedia.

[32]  Mei Han,et al.  Extract highlights from baseball game video with hidden Markov models , 2002, Proceedings. International Conference on Image Processing.