Automatic evaluation method for rushes summarization: Experimentation and analysis

Evaluation remains an important difficulty in the development of video summarization systems. Rigorous evaluation of video summaries generated by automatic systems is a complicated process because the ground truth is often difficult to define, and even when it exists, it is difficult to match with the obtain results. The TRECVID BBC evaluation campaign has recently introduced a rushes summarization task and has defined a manual evaluation methodology. In this paper, we explore the use of machine learning techniques to automate this evaluation. We present our approach and describe the current results, in comparison with manual evaluations performed in the 2007 campaign.

[1]  A. Murat Tekalp,et al.  Two-stage hierarchical video summary extraction to match low-level user browsing preferences , 2003, IEEE Trans. Multim..

[2]  Geoffrey I. Webb,et al.  Lazy Learning of Bayesian Rules , 2000, Machine Learning.

[3]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[4]  Ron Kohavi,et al.  The Power of Decision Tables , 1995, ECML.

[5]  Cuneyt M. Taskiran.  Evaluation of automatic video summarization systems , 2006, Electronic Imaging.

[6]  Ian H. Witten,et al.  Generating Accurate Rule Sets Without Global Optimization , 1998, ICML.

[7]  Michael G. Christel.  Evaluation and user studies with respect to video summarization and browsing , 2006, Electronic Imaging.

[8]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[9]  Ba Tu Truong,et al.  Video abstraction: A systematic review and classification , 2007, TOMCCAP.

[10]  Bernard Mérialdo,et al.  Split-screen dynamically accelerated video summaries , 2007, TVS '07.

[11]  S. Cessie,et al.  Ridge Estimators in Logistic Regression , 1992 .

[12]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[13]  John G. Cleary,et al.  K*: An Instance-based Learner Using and Entropic Distance Measure , 1995, ICML.

[14]  Robert C. Holte,et al.  Very Simple Classification Rules Perform Well on Most Commonly Used Datasets , 1993, Machine Learning.

[15]  Brian R. Gaines,et al.  Induction of ripple-down rules applied to modeling large databases , 1995, Journal of Intelligent Information Systems.

[16]  A. Murat Tekalp,et al.  Automatic Soccer Video Analysis and Summarization , 2003, IS&T/SPIE Electronic Imaging.

[17]  Kathleen R. McKeown,et al.  Summarization Evaluation Methods: Experiments and Analysis , 1998 .

[18]  Yoav Freund,et al.  The Alternating Decision Tree Learning Algorithm , 1999, ICML.

[19]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[20]  Ron Kohavi,et al.  Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid , 1996, KDD.

[21]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[22]  Wei-Hao Lin,et al.  Clever clustering vs. simple speed-up for summarizing rushes , 2007, TVS '07.

[23]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[24]  Paul Over,et al.  The trecvid 2007 BBC rushes summarization evaluation pilot , 2007, TVS '07.