Video parsing, retrieval and browsing: an integrated and content-based solution

This paper presents an integrated solution for computer assisted video parsing and content-based video retrieval and browsing. The uniqueness and effectiveness of this solution lies in its use of video content information provided by a parsing process driven by visual feature analysis. More specifically, parsing will temporally segment and abstract a video source, based on low-level image analyses; then retrieval and browsing of video will be based on key-frames selected during abstraction and spatial-temporal variations of visual features, as well as some shot-level semantics derived from camera operation and motion analysis. These processes, as well as video retrieval and browsing tools, are presented in detail as functions of an integrated system. Also, experimental results on automatic key-frame detection are given.

[1]  Boon-Lock Yeo,et al.  Rapid scene analysis on compressed video , 1995, IEEE Trans. Circuits Syst. Video Technol..

[2]  Akio Nagasaka,et al.  Automatic Video Indexing and Full-Video Search for Object Appearances , 1991, VDB.

[3]  Yihong Gong,et al.  Video parsing using compressed data , 1994, Electronic Imaging.

[4]  Ramesh C. Jain,et al.  Knowledge-guided parsing in video databases , 1993, Electronic Imaging.

[5]  Yihong Gong,et al.  An image database system with content capturing and fast image indexing abilities , 1994, 1994 Proceedings of IEEE International Conference on Multimedia Computing and Systems.

[6]  Anil K. Jain,et al.  Texture classification and segmentation using multiresolution simultaneous autoregressive models , 1992, Pattern Recognit..

[7]  Stephen W. Smoliar,et al.  Developing power tools for video indexing and retrieval , 1994, Electronic Imaging.

[8]  Anil K. Jain,et al.  Texture Analysis , 2018, Handbook of Image Processing and Computer Vision.

[9]  Walter Bender,et al.  Salient video stills: content and context preserved , 1993, MULTIMEDIA '93.

[10]  Alex Pentland,et al.  Photobook: tools for content-based manipulation of image databases , 1994, Other Conferences.

[11]  Remi Depommier,et al.  Content-based browsing of video sequences , 1994, MULTIMEDIA '94.

[12]  Brian C. O'Connor,et al.  Selecting Key Frames of Moving Image Documents: A Digital Environment for Analysis and Navigation. , 1991 .

[13]  C. H. Chen,et al.  Handbook of Pattern Recognition and Computer Vision , 1993 .

[14]  Marc Davis,et al.  Media Streams: an iconic visual language for video annotation , 1993, Proceedings 1993 IEEE Symposium on Visual Languages.

[15]  Markus A. Stricker,et al.  Similarity of color images , 1995, Electronic Imaging.

[16]  Hideyuki Tamura,et al.  Textural Features Corresponding to Visual Perception , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[17]  Michael Mills,et al.  A magnifier tool for video data , 1992, CHI.

[18]  Toshikazu Kato,et al.  A sketch retrieval method for full color image database-query by visual example , 1992, [1992] Proceedings. 11th IAPR International Conference on Pattern Recognition.

[19]  Makoto Miyahara,et al.  Mathematical Transform Of (R, G, B) Color Data To Munsell (H, V, C) Color Data , 1988, Other Conferences.

[20]  Rosalind W. Picard,et al.  Texture orientation for sorting photos "at a glance" , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[21]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Esther M. Arkin,et al.  An efficiently computable metric for comparing polygonal shapes , 1991, SODA '90.

[23]  John S. Boreczky,et al.  Indexes for user access to large video databases , 1994, Electronic Imaging.

[24]  Stephen W. Smoliar,et al.  Content-based video browsing tools , 1995, Electronic Imaging.

[25]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.