Video summarization with semantic concept preservation

A compelling video summarization should allow viewers to understand the summary content and recover the original plot correctly. To this end, we materialize the abstract elements that are cognitively informative for viewers as concepts. They implicitly convey the semantic structure and are instantiated by semantically redundant instances. Then we analyze that a good summary should i) keep various concepts complete and balanced so as to give viewers comparable cognitive clues from a complete perspective ii) pursue the most saliency so that the rendered summary is attractive to human perception. We then formulate video summarization as an integer programming problem and give a ranking based solution. We also propose a novel method to discover the latent concepts by spectral clustering of bag-of-words features. Experiment results on human evaluation scores demonstrate that our summarization approach performs well in terms of the informativeness, enjoyability and scalibility.

[1]  Michael R. Lyu,et al.  Semantic Video Summarization Using Mutual Reinforcement Principle and Shot Arrangement Patterns , 2005, 11th International Multimedia Modelling Conference.

[2]  Yue Gao,et al.  Video summarization by redundancy removing and content ranking , 2007, ACM Multimedia.

[3]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  Chong-Wah Ngo,et al.  Video summarization and scene detection by graph modeling , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Yu Huang,et al.  Video retargeting with nonlinear spatial-temporal saliency fusion , 2010, 2010 IEEE International Conference on Image Processing.

[6]  Aggelos K. Katsaggelos,et al.  MINMAX optimal video summarization , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  José María Martínez Sanchez,et al.  A Framework for Scalable Summarization of Video , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Jhing-Fa Wang,et al.  A Novel Video Summarization Based on Mining the Story-Structure and Semantic Relations Among Concept Entities , 2009, IEEE Transactions on Multimedia.

[9]  Lie Lu,et al.  A generic framework of user attention model and its application in video summarization , 2005, IEEE Trans. Multim..

[10]  David S. Doermann,et al.  Video summarization by curve simplification , 1998, MULTIMEDIA '98.

[11]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..