A Survey of Technologies for Parsing and Indexing Digital Video1

Abstract In the future we.envision systems that will provide video information delivery services to customers on a very large scale. These systems must provide customers with mechanisms to select programs of their choice from live broadcasts. Customers should also be provided with easy means of browsing and accessing prerecorded digital data (e.g., distributed digital multimedia libraries), and downloading data from other information sources. To be viable for such large information sets, these systems must understand customer preferences and tailor the available information to the customer's needs. To support this vision, a number of issues must be addressed and obstacles overcome. Intuitive interfaces, powerful query formulation and evaluation techniques, comprehensive data models, and flexible presentation functionalities must be developed. To realize these components, an effective query evaluation engine with the capabilities of query resolution in different content-specific formats (e.g., by graphics, by image, by sound) and in different domain-specific models (e.g., database of movies, database of newsclips) should be present. Additionally, the digital video database will require an efficient indexing system for easy access to the stored information. In this paper we discuss existing research trends in this area and requirements for future data delivery systems. An overview of video indexing is presented followed by a discussion on current indexing techniques.

[1]  Vishvjit S. Nalwa,et al.  A guided tour of computer vision , 1993 .

[2]  SUH-YIN LEE,et al.  Access Methods of Image Database , 1990, Int. J. Pattern Recognit. Artif. Intell..

[3]  Christos Faloutsos,et al.  An Artificial Pictorial Database System for PQSL , 1988 .

[4]  Rudolf Bayer,et al.  Prefix B-trees , 1977, TODS.

[5]  Michael Stonebraker,et al.  Rule indexing implementations in database systems , 1986 .

[6]  Alberto Del Bimbo,et al.  A Three-Dimensional Iconic Environment for Image Database Querying , 1993, IEEE Trans. Software Eng..

[7]  T. Hamano,et al.  A similarity retrieval method for image databases using simple graphics , 1988, [Proceedings] 1988 IEEE Workshop on Languages for Automation@m_Symbiotic and Intelligent Robotics.

[8]  Thomas D. C. Little,et al.  Video scene decomposition with the motion picture parser , 1994, Electronic Imaging.

[9]  Shih-Fu Chang,et al.  Scene change detection in an MPEG-compressed video sequence , 1995, Electronic Imaging.

[10]  Hans-Peter Kriegel,et al.  The R*-tree: an efficient and robust access method for points and rectangles , 1990, SIGMOD '90.

[11]  Shi-Kuo Chang,et al.  Iconic Indexing by 2-D Strings , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Frank Manola,et al.  PROBE Spatial Data Modeling and Query Processing in an Image Database Application , 1988, IEEE Trans. Software Eng..

[13]  Glorianna Davenport,et al.  Cinematic primitives for multimedia , 1991, IEEE Computer Graphics and Applications.

[14]  Ramesh C. Jain,et al.  Segmentation of Frame Sequences Obtained by a Moving Observer , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Shi-Kuo Chang Visual reasoning for information retrieval from very large databases , 1990, J. Vis. Lang. Comput..

[16]  Tosiyasu L. Kunii,et al.  Pictorial Data-Base Systems , 1981, Computer.

[17]  William I. Grosky,et al.  A pictorial index mechanism for model-based matching , 1992, Data Knowl. Eng..

[18]  Thomas Joseph,et al.  PICQUERY: A High Level Query Language for Pictorial Database Management , 1988, IEEE Trans. Software Eng..

[19]  Yoshinobu Tonomura,et al.  Projection Detecting Filter for Video Cut Detection , 1993, ACM Multimedia.

[20]  Jungwoo Lee,et al.  Multiresolution video indexing for subband coded video databases , 1994, Electronic Imaging.

[21]  Dinesh Venkatesh,et al.  SELECTION AND DISSEMINATION OF DIGITAL VIDEO VIA THE VIRTUAL VIDEO BROWSER , 1995 .

[22]  Vijay V. Raghavan,et al.  A Unified Approach to Data Modeling and Retrieval for a Class of Image Database Applications , 1996, Multimedia Database System: Issues and Research Direction.

[23]  Ramana Rao,et al.  Rich interaction in the digital library , 1995, CACM.

[24]  S. Loeb,et al.  Lessons from Lyrictime: A Prototype Multimedia System , 1992, 4th IEEE ComSoc International Workshop on Multimedia Communications. MULTIMEDIA.

[25]  Marc Davis,et al.  Media Streams: an iconic visual language for video annotation , 1993, Proceedings 1993 IEEE Symposium on Visual Languages.

[26]  Information technology — Generic coding of moving pictures and associated audio information — Part 2 : Video Technologies , 2022 .

[27]  Hanan Samet,et al.  The Design and Analysis of Spatial Data Structures , 1989 .

[28]  Shi-Kuo Chang Visual reasoning for informational retrieval from very large databases , 1989, [Proceedings] 1989 IEEE Workshop on Visual Languages.

[29]  Jürg Nievergelt,et al.  The Grid File: An Adaptable, Symmetric Multikey File Structure , 1984, TODS.

[30]  Arding Hsu,et al.  Image processing on compressed data for large video databases , 1993, MULTIMEDIA '93.

[31]  Suh-Yin Lee,et al.  Video indexing: an approach based on moving object and track , 1993, Electronic Imaging.

[32]  King-Sun Fu,et al.  Picture Query Languages for Pictorial Data-Base Systems , 1981, Computer.

[33]  Gregory K. Wallace,et al.  The JPEG still picture compression standard , 1992 .

[34]  Myron Flickner,et al.  Query by Image and Video Content , 1995 .

[35]  Carlo Meghini,et al.  Conceptual modeling of multimedia documents , 1991, Computer.

[36]  Hans-Hellmut Nagel,et al.  Formation of an object concept by analysis of systematic time variations in the optically perceptible environment , 1978 .

[37]  Y. Li,et al.  Representation of multi-resolution symbolic and binary pictures using 2D H-strings , 1988, [Proceedings] 1988 IEEE Workshop on Languages for Automation@m_Symbiotic and Intelligent Robotics.

[38]  Masahito Hirakawa,et al.  IconicBrowser: An Iconic Retrieval System for Object-Oriented Databases , 1990, J. Vis. Lang. Comput..

[39]  Ramin Zabih,et al.  A feature-based algorithm for detecting and classifying scene breaks , 1995, MULTIMEDIA '95.

[40]  Franz Aurenhammer,et al.  Voronoi diagrams—a survey of a fundamental geometric data structure , 1991, CSUR.

[41]  Ramesh C. Jain,et al.  Digital video segmentation , 1994, MULTIMEDIA '94.

[42]  Walter Bender,et al.  Newspace: Mass Media and Personal Computing , 1991, USENIX Summer.

[43]  Thomas D. C. Little,et al.  Video query formulation , 1995, Electronic Imaging.

[44]  Richard A. Gustafson Elements of the randomized combinatorial file structure , 1971, SIGIR '71.

[45]  Ramesh C. Jain,et al.  Dynamic vision , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[46]  Alfonso F. Cardenas,et al.  Database Structure and Manipulation Capabilities of a Picture Database Management System (PICDMS) , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Claude L. Fennema,et al.  Velocity determination in scenes containing several moving objects , 1979 .

[48]  Toshikazu Kato,et al.  Query by Visual Example - Content based Image Retrieval , 1992, EDBT.

[49]  H. V. Jagadish,et al.  A retrieval technique for similar shapes , 1991, SIGMOD '91.

[50]  Glorianna Davenport,et al.  The Stratification System - A Design Emvironment for Random Access , 1992, NOSSDAV.

[51]  Christos Faloutsos,et al.  Signature files: design and performance comparison of some signature extraction methods , 1985, SIGMOD Conference.

[52]  John S. Boreczky,et al.  Indexes for user access to large video databases , 1994, Electronic Imaging.

[53]  Antonin Guttman,et al.  R-trees: a dynamic index structure for spatial searching , 1984, SIGMOD '84.

[54]  Douglas Comer,et al.  Ubiquitous B-Tree , 1979, CSUR.

[55]  Guy M. Lohman,et al.  Differential files: their application to the maintenance of large databases , 1976, TODS.

[56]  Natalio Pincever,et al.  Parsing Movies in Context , 1991, USENIX Summer.

[57]  J. T. Robinson,et al.  The K-D-B-tree: a search structure for large multidimensional dynamic indexes , 1981, SIGMOD '81.

[58]  Behzad Shahraray,et al.  Scene change detection and content-based sampling of video sequences , 1995, Electronic Imaging.

[59]  CORPORATE The Stanford Digital Libraries Group The Stanford Digital Library Project , 1995, CACM.

[60]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[61]  Gerard Salton,et al.  Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer , 1989 .

[62]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[63]  Christos Faloutsos,et al.  Access methods for text , 1985, CSUR.

[64]  Christos Faloutsos,et al.  A Multimedia Office Filing System , 1983, VLDB.

[65]  S. Loeb,et al.  Delivering interactive multimedia documents over networks , 1992, IEEE Communications Magazine.

[66]  Christos Faloutsos,et al.  Description and performance analysis of signature file methods for office filing , 1987, TOIS.

[67]  Akio Nagasaka,et al.  Automatic Video Indexing and Full-Video Search for Object Appearances , 1991, VDB.

[68]  Yihong Gong,et al.  Video parsing using compressed data , 1994, Electronic Imaging.

[69]  Michael Stonebraker,et al.  Chabot: Retrieval from a Relational Database of Images , 1995, Computer.