A Survey on the Automatic Indexing of Video Data,

Today a considerable amount of video data in multimedia databases requires sophisticated indices for its effective use. Manual indexing is the most effective method to do this, but it is also the slowest and the most expensive. Automated methods have then to be developed. This paper surveys several approaches and algorithms that have been recently proposed to automatically structure audio?visual data, both for annotation and access.

[1]  Andrew S. Gordon,et al.  Conceptual Indexing for Video Retrieval , 1995 .

[2]  Edward J. Delp,et al.  A fast algorithm for video parsing using MPEG compressed sequences , 1995, Proceedings., International Conference on Image Processing.

[3]  Nilesh V. Patel,et al.  Video shot detection and characterization for video databases , 1997, Pattern Recognit..

[4]  Ullas Gargi,et al.  Evaluation of video sequence indexing and hierarchical video indexing , 1995, Electronic Imaging.

[5]  P. Venkat Rangan,et al.  Multimedia Storage Servers: A Tutorial , 1995, Computer.

[6]  Christos Faloutsos,et al.  FastMap: a fast algorithm for indexing, data-mining and visualization of traditional and multimedia datasets , 1995, SIGMOD '95.

[7]  Marco La Cascia,et al.  A Real-Time Neural Approach to Scene Cut Detection , 1996 .

[8]  Ramesh Jain,et al.  Storage and Retrieval for Image and Video Databases III , 1995 .

[9]  Mubarak Shah,et al.  Motion-based recognition a survey , 1995, Image Vis. Comput..

[10]  Haruhiko Nishiyama,et al.  An image retrieval system considering subjective perception , 1994, CHI '94.

[11]  V. S. Subrahmanian,et al.  Heterogeneous Multimedia Reasoning , 1995, Computer.

[12]  K. Rijkse,et al.  H.263: video coding for low-bit-rate communication , 1996, IEEE Commun. Mag..

[13]  Gilles Burel,et al.  Detection and localization of faces on digital images , 1994, Pattern Recognit. Lett..

[14]  Michael Hawley.  Structure out of sound , 1993 .

[15]  Michael G. Christel,et al.  Evolving video skims into useful multimedia abstractions , 1998, CHI.

[16]  Walter Bender,et al.  Salient stills , 1992, CHI '92.

[17]  Forouzan Golshani,et al.  Motion recovery for video content classification , 1995, TOIS.

[18]  Didier Le Gall,et al.  MPEG: a video compression standard for multimedia applications , 1991, CACM.

[19]  Edward M. Riseman,et al.  Finding text in images , 1997, DL '97.

[20]  Takeo Kanade,et al.  Video OCR: indexing digital news libraries by recognition of superimposed captions , 1999, Multimedia Systems.

[21]  Boon-Lock Yeo,et al.  Analysis And Presentation Of Soccer Highlights From Digital Video , 1995 .

[22]  V. S. Subrahmanian,et al.  Multimedia Database Systems , 1993, Artificial Intelligence.

[23]  Philippe Aigrain,et al.  The automatic real-time analysis of film editing and transition effects and its applications , 1994, Comput. Graph..

[24]  Thomas D. C. Little,et al.  A Survey of Technologies for Parsing and Indexing Digital Video1 , 1996, J. Vis. Commun. Image Represent..

[25]  Ramesh C. Jain,et al.  Digital video segmentation , 1994, MULTIMEDIA '94.

[26]  Shih-Fu Chang,et al.  VideoQ: an automated content based video search system using visual cues , 1997, MULTIMEDIA '97.

[27]  Fillia Makedon,et al.  Automatic Video Pause Detection Filter , 1997 .

[28]  Thomas D. C. Little,et al.  Video query formulation , 1995, Electronic Imaging.

[29]  Boon-Lock Yeo,et al.  Extracting story units from long programs for video browsing and navigation , 1996, Proceedings of the Third IEEE International Conference on Multimedia Computing and Systems.

[30]  Wayne H. Wolf,et al.  Key frame selection by motion analysis , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[31]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Patrick M. Kelly,et al.  Experience with CANDID: comparison algorithm for navigating digital image databases , 1995, Other Conferences.

[33]  Minerva M. Yeung,et al.  Efficient matching and clustering of video shots , 1995, Proceedings., International Conference on Image Processing.

[34]  David S. Doermann,et al.  Video summarization by curve simplification , 1998, MULTIMEDIA '98.

[35]  Ullas Gargi,et al.  Performance characterization and comparison of video indexing algorithms , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[36]  Wolfgang Effelsberg,et al.  Abstracting Digital Movies Automatically , 1996, J. Vis. Commun. Image Represent..

[37]  Alex Pentland,et al.  Probabilistic Visual Learning for Object Representation , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  M. Bierling,et al.  Displacement Estimation By Hierarchical Blockmatching , 1988, Other Conferences.

[39]  Boon-Lock Yeo,et al.  Video browsing using clustering and scene transitions on compressed sequences , 1995, Electronic Imaging.

[40]  Michael A. Smith,et al.  Video skimming and characterization through the combination of image and language understanding techniques , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[41]  Glorianna Davenport,et al.  ConText towards the evolving documentary , 1995, MULTIMEDIA '95.

[42]  Ullas Gargi,et al.  Semiautomatic video database system , 1995, Electronic Imaging.

[43]  John Wang,et al.  Applying mid-level vision techniques for video data compression and manipulation , 1994, Electronic Imaging.

[44]  Suh-Yin Lee,et al.  Video indexing: an approach based on moving object and track , 1993, Electronic Imaging.

[45]  P. Venkat Rangan,et al.  Multimedia Storage Servers: A Tutorial , 1995, Computer.

[46]  Max Mühlhäuser,et al.  OBVI: Hierarchical 3D Video-Browsing , 1998 .

[47]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[48]  Edward A. Fox,et al.  Advances in interactive digital multimedia systems , 1991, Computer.

[49]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[50]  Nilesh V. Patel,et al.  Statistical approach to scene change detection , 1995, Electronic Imaging.

[51]  Hain-Ching Liu,et al.  Automatic determination of scene changes in MPEG compressed video , 1995, Proceedings of ISCAS'95 - International Symposium on Circuits and Systems.

[52]  Wolfgang Effelsberg,et al.  VisualGREP: a systematic method to compare and retrieve video sequences , 1997, Electronic Imaging.

[53]  Ramesh C. Jain,et al.  Feature Based Digital Video Indexing , 1997, VDB.

[54]  Stephen W. Smoliar,et al.  Content-based video browsing tools , 1995, Electronic Imaging.

[55]  Michael S. Lew,et al.  Information theory and face detection , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[56]  Patrick M. Kelly,et al.  CANDID: comparison algorithm for navigating digital image databases , 1994, Seventh International Working Conference on Scientific and Statistical Database Management.

[57]  Ramin Zabih,et al.  A feature-based algorithm for detecting and classifying scene breaks , 1995, MULTIMEDIA '95.

[58]  Boon-Lock Yeo,et al.  Rapid scene analysis on compressed video , 1995, IEEE Trans. Circuits Syst. Video Technol..

[59]  Arding Hsu,et al.  Feature management for large video databases , 1993, Electronic Imaging.

[60]  Shih-Fu Chang,et al.  Scene change detection in an MPEG-compressed video sequence , 1995, Electronic Imaging.

[61]  Choong Woong Lee,et al.  Robust estimation of camera parameters from image sequence for video composition , 1996, Signal Process. Image Commun..

[62]  Takeo Kanade,et al.  Semantic analysis for video contents extraction—spotting by association in news video , 1997, MULTIMEDIA '97.

[63]  Don R. Hush,et al.  Query by image example: The CANDID approach , 1995 .

[64]  Shih-Fu Chang,et al.  A highly efficient system for automatic face region detection in MPEG video , 1997, IEEE Trans. Circuits Syst. Video Technol..

[65]  A. Desai Narasimhalu,et al.  Multimedia databases , 1996, Multimedia Systems.

[66]  Wei Xiong,et al.  Efficient Scene Change Detection and Camera Motion Annotation for Video Classification , 1998, Comput. Vis. Image Underst..

[67]  Edoardo Ardizzone,et al.  Video indexing using optical flow field , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[68]  Wolfgang Effelsberg,et al.  Automatic recognition of film genres , 1995, MULTIMEDIA '95.

[69]  Shih-Fu Chang,et al.  Video object model and segmentation for content-based video indexing , 1997, Proceedings of 1997 IEEE International Symposium on Circuits and Systems. Circuits and Systems in the Information Age ISCAS '97.

[70]  Yasuo Ariki,et al.  Indexing and classification of TV news articles based on telop recognition , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[71]  Hae-Kwang Kim,et al.  Efficient Automatic Text Location Method and Content-Based Indexing and Structuring of Video Database , 1996, J. Vis. Commun. Image Represent..

[72]  Anil K. Jain,et al.  Automatic text location in images and video frames , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[73]  Yoshinobu Tonomura,et al.  VideoMAP and VideoSpaceIcon: tools for anatomizing video content , 1993, INTERCHI.

[74]  Christian Weiser.  Multimedia Information Systems for Hotel Guests , 1995 .

[75]  Atreyi Kankanhalli,et al.  A Video Database System for Digital Libraries , 1994, DL.

[76]  Harpreet S. Sawhney,et al.  Compact Representations of Videos Through Dominant and Multiple Motion Estimation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[77]  Roberto Brunelli,et al.  MASSACHUSETTS INSTITUTE OF TECHNOLOGY ARTIFICIAL INTELLIGENCE LABORATORY and CENTER FOR BIOLOGICAL AND COMPUTATIONAL LEARNING DEPARTMENT OF BRAIN AND COGNITIVE SCIENCES , 2001 .

[78]  Philippe Aigrain,et al.  Medium knowledge-based macro-segmentation of video into sequences , 1997 .

[79]  Takeo Kanade,et al.  Name-It: Naming and Detecting Faces in Video by the Integration of Image and Natural Language Processing , 1997, IJCAI.

[80]  Philippe Joly,et al.  Efficient automatic analysis of camera work and microsegmentation of video using spatiotemporal images , 1996, Signal Process. Image Commun..

[81]  Ahmed Karmouch,et al.  Detecting Cuts by Understanding Camera Operations for Video Indexing , 1995, J. Vis. Lang. Comput..

[82]  Wolfgang Effelsberg,et al.  Automatic text segmentation and text recognition for video indexing , 2000, Multimedia Systems.

[83]  Hong Heather Yu,et al.  Scenic classification methods for image and video databases , 1995, Other Conferences.

[84]  Edward H. Adelson,et al.  Spatio-temporal segmentation of video data , 1994, Electronic Imaging.

[85]  Boon-Lock Yeo,et al.  A unified approach to temporal segmentation of motion JPEG and MPEG compressed video , 1995, Proceedings of the International Conference on Multimedia Computing and Systems.

[86]  Harpreet S. Sawhney,et al.  Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding , 1995, Proceedings of IEEE International Conference on Computer Vision.

[87]  Lawrence A. Rowe,et al.  Multimedia systems and applications , 2010, 2010 International Conference on Signal Processing and Multimedia Applications (SIGMAP).

[88]  Harpreet S. Sawhney,et al.  Model-based 2D&3D dominant motion estimation for mosaicing and video representation , 1995, Proceedings of IEEE International Conference on Computer Vision.

[89]  John S. Boreczky,et al.  Comparison of video shot boundary detection techniques , 1996, Electronic Imaging.

[90]  Alberto Del Bimbo,et al.  Commercial video retrieval by induced semantics , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[91]  Alberto Del Bimbo,et al.  Film Editing Reconstruction and Semantic Analysis , 1995, CAIP.

[92]  Tomaso A. Poggio,et al.  Example-Based Learning for View-Based Human Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[93]  Stefano Messelodi,et al.  Automatic identification and skew estimation of text lines in real scene images , 1999, Pattern Recognition.

[94]  Wei Xiong,et al.  Automatic video data structuring through shot partitioning and key-frame computing , 1997, Machine Vision and Applications.

[95]  Thomas D. C. Little,et al.  Video scene decomposition with the motion picture parser , 1994, Electronic Imaging.