F-divergences driven video key frame extraction

This paper proposes a shot-based key frame selection technique, aiming at generating a condensed set of frames representing the essential content of a video sequence. Inspired by the successful utilization of Jensen-Shannon Divergence (JSD) and Jensen-Rényi Divergence (JRD) for key frame selection [1] [2], we investigate several popularly accepted f-divergences for calculating the frame-by-frame distance to segment the video clip and then to obtain the key frames. Based on simulation and real test videos, the performances of the key frame selection method by using different versions of f-divergences are systematically analyzed. Extensive experimentation shows that, compared with the methods using JSD and JRD, the new technique is slightly better and computationally faster.

[1]  Ioannis Pitas,et al.  Information theory-based shot cut/fade detection and video summarization , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Klaus Schöffmann,et al.  Mobile video browsing with a 3D filmstrip , 2013, ICMR '13.

[3]  John R. Kender,et al.  Computational approaches to temporal sampling of video sequences , 2007, TOMCCAP.

[4]  K. Pearson.  On the Criterion that a Given System of Deviations from the Probable in the Case of a Correlated System of Variables is Such that it Can be Reasonably Supposed to have Arisen from Random Sampling , 1900 .

[5]  Alexandra Psarrou,et al.  Key-frame extraction algorithm using entropy difference , 2004, MIR '04.

[6]  Sang Uk Lee,et al.  Efficient video indexing scheme for content-based retrieval , 1999, IEEE Trans. Circuits Syst. Video Technol..

[7]  S. S. Dragomir,et al.  SOME INEQUALITIES FOR THE CSISZÁR Φ-DIVERGENCE , 2003 .

[8]  Flemming Topsøe,et al.  Some inequalities for information divergence and related measures of discrimination , 2000, IEEE Trans. Inf. Theory.

[9]  Mateu Sbert,et al.  Selection and 3D visualization of video key frames , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[10]  S. M. Ali,et al.  A General Class of Coefficients of Divergence of One Distribution from Another , 1966 .

[11]  Karl Pearson F.R.S..  X. On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling , 2009 .

[12]  Klaus Schöffmann,et al.  Mobile video browsing with the ThumbBrowser , 2013, MM '13.

[13]  Dominik Endres,et al.  A new metric for probability distributions , 2003, IEEE Transactions on Information Theory.

[14]  E. Hellinger,et al.  Neue Begründung der Theorie quadratischer Formen von unendlichvielen Veränderlichen. , 1909 .

[15]  Mateu Sbert,et al.  Key frame selection based on Jensen-Rényi divergence , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[16]  Xiao-Ping Zhang,et al.  Video shot boundary detection using independent component analysis , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[17]  Boon-Lock Yeo,et al.  Rapid scene analysis on compressed video , 1995, IEEE Trans. Circuits Syst. Video Technol..

[18]  Mona Omidyeganeh,et al.  Video Keyframe Analysis Using a Segment-Based Statistical Metric in a Visually Sensitive Parametric Space , 2011, IEEE Trans. Image Process..