Real-Time Storyboard Generation in Videos Using a Probability Distribution Based Threshold

Storyboard generation in real-time is a task which aims at producing a meaningful summary of a video in a duration which is comparable to its length. The main challenge in real-time videos is to extract a set of key-frames on the fly, without having access to the full content. A fast and accurate generation of a dynamic threshold is important to ensure that the summary produced is a succinct representation of the video and appropriate in terms of content coverage. In this work, an efficient method has been developed for producing a static summary using a probability distribution based threshold. An open framework has been used in the redundancy reduction phase which may incorporate any of the state-of-the-art similarity matching techniques so as to eliminate the superfluous key-frames. The efficacy of the proposed method is depicted through the high recall and precision values obtained from the experimental results on several publicly available videos of different genre and length. The method has been tested on computers having different system configurations, thereby proving the efficacy of the proposed technique in real-time.

[1]  Michael A. Smith,et al.  Video skimming and characterization through the combination of image and language understanding techniques , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Nathalie Guyader,et al.  Video Summarization Based on Camera Motion and a Subjective Evaluation Method , 2007, EURASIP J. Image Video Process..

[3]  Guillermo Cámara Chávez,et al.  A New Method for Static Video Summarization Using Local Descriptors and Video Temporal Segmentation , 2013, 2013 XXVI Conference on Graphics, Patterns and Images.

[4]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[5]  Siddhartha Bhattacharyya,et al.  An Unsupervised Method for Real Time Video Shot Segmentation , 2014 .

[6]  Siddhartha Bhattacharyya,et al.  Enhancement of perceptual quality in static video summarization using minimal spanning tree approach , 2015, 2015 IEEE International Conference on Signal Processing, Informatics, Communication and Energy Systems (SPICES).

[7]  Xavier Binefa,et al.  An EM algorithm for video summarization, generative model approach , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[8]  Sang Uk Lee,et al.  Efficient video indexing scheme for content-based retrieval , 1999, IEEE Trans. Circuits Syst. Video Technol..

[9]  Narayanaswamy Balakrishnan,et al.  Order statistics and inference , 1991 .

[10]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[11]  R. Narasimha,et al.  Key frame extraction using MPEG-7 motion descriptors , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[12]  Jan-Michael Frahm,et al.  Modeling and Recognition of Landmark Image Collections Using Iconic Scene Graphs , 2008, International Journal of Computer Vision.

[13]  B. B. Meshram,et al.  Content based video retrieval systems , 2012, ArXiv.

[14]  Santosh S. Vempala,et al.  Latent semantic indexing: a probabilistic analysis , 1998, PODS '98.

[15]  Jean-Nicolas Ouellet,et al.  To Watch or Not to Watch: Video Summarization with Explicit Duplicate Elimination , 2011, 2011 Canadian Conference on Computer and Robot Vision.

[16]  Alan Hanjalic,et al.  An integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis , 1999, IEEE Trans. Circuits Syst. Video Technol..

[17]  Haibin Liu,et al.  Video linkage: group based copied video detection , 2008, CIVR '08.

[18]  HongJiang Zhang,et al.  A model of motion attention for video skimming , 2002, Proceedings. International Conference on Image Processing.

[19]  Sinisa Segvic,et al.  Classifying Traffic Scenes Using The GIST Image Descriptor , 2013, ArXiv.

[20]  Siddhartha Bhattacharyya,et al.  Video Shot Segmentation Using Spatio-temporal Fuzzy Hostility Index and Automatic Threshold , 2014, 2014 Fourth International Conference on Communication Systems and Network Technologies.

[21]  B. B. Meshram,et al.  Content based video retrieval , 2012, ArXiv.

[22]  Alexei A. Efros,et al.  Scene completion using millions of photographs , 2007, SIGGRAPH 2007.

[23]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[24]  Siddhartha Bhattacharyya,et al.  Towards redundancy reduction in storyboard representation for static video summarization , 2014, 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[25]  Nuno Vasconcelos,et al.  A spatiotemporal motion model for video summarization , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[26]  Rainer Lienhart Dynamic video summarization of home video , 1999, Electronic Imaging.

[27]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.