Performance Evaluation of Shot Boundary Detection Metrics in the Presence of Object and Camera Motion

Abstract Partitioning a video into shots is an important step for video indexing. We have presented the performance of various traditional metrics that are generally used to detect shot boundaries. In this paper, we evaluated shot boundary detection metrics, such as likelihood ratio and color ratio histogram in Red Green Blue (RGB) and Hue Saturation, Value (HSV) color space for three different action and thriller movies. These movies consist of large number of frames with object and camera motion. The pixel difference and Chi-square shot boundary detection metrics in Luma and Chrominance Components (YUV) color space has been tested for Ave different movies. The results were evaluated in terms of Recall, Precision, and F1 measure for all these movies. It has been observed that these results are affected by the disturbance due to the motion in the consecutive frames. The false positives and miss detection of shot boundaries in all the tested metrics are due to fast camera and object motion. An algorithm has been proposed for shot boundary detection by using dual tree complex wavelet transform in the presence of motion. Performance comparison of the proposed algorithm with the traditional metrics validates its effectiveness in terms of improved Recall, Precision, and F1 score.

[1]  Nilesh V. Patel,et al.  Statistical approach to scene change detection , 1995, Electronic Imaging.

[2]  Nick Kingsbury,et al.  The dual-tree complex wavelet transform: a new technique for shift invariance and directional filters , 1998 .

[3]  Bo Zhang,et al.  A Formal Study of Shot Boundary Detection , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Ullas Gargi,et al.  Performance characterization of video-shot-change detection methods , 2000, IEEE Trans. Circuits Syst. Video Technol..

[5]  Rainer Lienhart,et al.  Comparison of automatic shot boundary detection algorithms , 1998, Electronic Imaging.

[6]  Atreyi Kankanhalli,et al.  Automatic partitioning of full-motion video , 1993, Multimedia Systems.

[7]  Ralph M. Ford,et al.  Metrics for shot boundary detection in digital video sequences , 2000, Multimedia Systems.

[8]  N. Kingsbury.  Image processing with complex wavelets , 1999, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[9]  Alan Hanjalic,et al.  Shot-boundary detection: unraveled and resolved? , 2002, IEEE Trans. Circuits Syst. Video Technol..

[10]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[11]  Zhou Wang,et al.  Translation insensitive image similarity in complex wavelet domain , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[12]  Rangachar Kasturi,et al.  Machine vision , 1995 .

[13]  F. Arman,et al.  A Statistical Approach to Scene Change Detection , 1995 .

[14]  John S. Boreczky,et al.  Comparison of video shot boundary detection techniques , 1996, J. Electronic Imaging.

[15]  Richard Baraniuk,et al.  The dual-tree complex wavelet transform , 2005, IEEE Signal Processing Magazine.

[16]  NagasakaAkio,et al.  Automatic video indexing and full-video search for object appearances (abstract) , 1992 .

[17]  Uday B. Desai,et al.  Shot boundary detection in the presence of fire flicker and explosion using stationary wavelet transform , 2011, Signal Image Video Process..