A Video Shot Boundary Detection Approach Based on CNN Feature

In nowadays, as the development of digital photographic technology, video files grow rapidly, there is a great demand for automatic video semantic analysis in many scenes, such as video semantic understanding, content-based analysis, video retrieval. Shot boundary detection is a key basic technology and first step for video analysis. However, recent methods are time consuming and performs bad in the gradual transition detection. In this paper we proposed a new approach which used CNN model to extract features of video sequence parallelly based on GPU, so we can simplify the expression of video and reduce the calculation time for shot detection, and took local frame similarity and dual-threshold sliding window similarity into consideration to increase recall and precise of shot detection. The experimental result shows that the proposed method can achieve a high F1 score and excellent detection speed.

[1]  Ramin Zabih,et al.  A feature-based algorithm for detecting and classifying production effects , 1999, Multimedia Systems.

[2]  Ioannis Pitas,et al.  Video shot-boundary detection using singular-value decomposition and statistical tests , 2007, J. Electronic Imaging.

[3]  Yiannis Kompatsiaris,et al.  Gradual transition detection using color coherence and other criteria in a video shot meta-segmentation framework , 2008, 2008 15th IEEE International Conference on Image Processing.

[4]  Xiangyang Xue,et al.  Shot Boundary Classification and Refinement Using Inter-Frame Similarity Patterns , 2005, 2005 5th International Conference on Information Communications & Signal Processing.

[5]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Y.-N. Li,et al.  Fast video shot boundary detection framework employing pre-processing techniques , 2009, IET Image Process..

[8]  Alan Hanjalic,et al.  Shot-boundary detection: unraveled and resolved? , 2002, IEEE Trans. Circuits Syst. Video Technol..

[9]  Yong Shi,et al.  Fast Video Shot Boundary Detection Based on SVD and Pattern Matching , 2013, IEEE Transactions on Image Processing.

[10]  S. Domnic,et al.  Video cut detection using block based histogram differences in RGB color space , 2010, 2010 International Conference on Signal and Image Processing.

[11]  Ananya SenGupta A Formal Study of Video Segmentation , 2015 .

[12]  Yi Wan,et al.  A novel metric for efficient video shot boundary detection , 2014, 2014 IEEE Visual Communications and Image Processing Conference.

[13]  Rita Cucchiara,et al.  Linear Transition Detection as a Unified Shot Detection Approach , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Bo Zhang,et al.  A Formal Study of Shot Boundary Detection , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Ioannis Pitas,et al.  Information theory-based shot cut/fade detection and video summarization , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Hou Guang-hua The research of video segmentation and scene clustering algorithms , 2006 .