Foundations of multimedia database systems

Though numerous multimedia systems exist in the commercial market today, relatively little work has been done on developing the mathematical foundation of multimedia technology. We attempt to take some initial steps towards the development of a theoretical basis for a multimedia information system. To do so, we develop the motion of a structured multimedia database system. We begin by defining a mathematical model of a media-instance. A media-instance may be thought of as “glue” residing on top of a specific physical media-representation (such as video, audio, documents, etc). Using this “glue”, it is possible to define a general purpose logical query language to query multimedia data. This glue consists of a set of “states” (e.g., video frames, audio tracks, etc.) and “features”, together with relationships between states and/or features. A structured multimedia database system imposes a certain mathematical structures on the set of features/states. Using this notion of a structure, we are able to define indexing structures for processing queries, methods to relax queries when answers do not exist to those queries, as well as sound, complete and terminating procedures to answer such queries (and their relaxations, when appropriate). We show how a media-presentation can be generated by processing a sequence of queries, and furthermore we show that when these queries are extended to include constraints, then these queries can not only generate presentations, but also generate temporal synchronization properties and spatial layout properties for such presentations. We describe the architecture of a prototype multimedia database system based on the principles described in this paper.

[1]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[2]  Roland H. C. Yap,et al.  The CLP( R ) language and system , 1992, TOPL.

[3]  Yihong Gong,et al.  An image database system with content capturing and fast image indexing abilities , 1994, 1994 Proceedings of IEEE International Conference on Multimedia Computing and Systems.

[4]  Katsumi Tanaka,et al.  OVID: Design and Implementation of a Video-Object Database System , 1993, IEEE Trans. Knowl. Data Eng..

[5]  Clement T. Yu,et al.  Reasoning About Spatial Relationships in Picture Retrieval Systems , 1994, VLDB.

[6]  Terry E. Weymouth,et al.  Semantic Queries with Pictures: The VIMSYS Model , 1991, VLDB.

[7]  Dennis Tsichritzis,et al.  Data modeling of time-based media , 1994, SIGMOD '94.

[8]  Anil Nerode,et al.  Mixed integer programming methods for computing nonmonotonic deductive databases , 1994, JACM.

[9]  V. S. Subrahmanian,et al.  Amalgamating Knowledge Bases, II: Distributed Mediators , 1994, Int. J. Cooperative Inf. Syst..

[10]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[11]  V. S. Subrahmanian,et al.  Theory of Generalized Annotated Logic Programming and its Applications , 1992, J. Log. Program..

[12]  V. S. Subrahmanian,et al.  Using hybrid knowledge bases for missile siting problems , 1994, Proceedings of the Tenth Conference on Artificial Intelligence for Applications.

[13]  V. S. Subrahmanian,et al.  Querying Video Libraries* , 1996, J. Vis. Commun. Image Represent..

[14]  William I. Grosky,et al.  Towards a data model for integrated pictorial databases , 1983, Comput. Vis. Graph. Image Process..

[15]  Joxan Jaffar,et al.  Constraint logic programming , 1987, POPL '87.

[16]  V. S. Subrahmanian,et al.  Paraconsistent Logic Programming , 1987, Theor. Comput. Sci..

[17]  Shahram Ghandeharizadeh,et al.  Staggered striping in multimedia information systems , 1994, SIGMOD '94.

[18]  William I. Grosky,et al.  Multimedia information systems , 1994, IEEE MultiMedia.

[19]  Ricky K. Taira,et al.  The Knowledge-Based Object-Oriented PICQUERY+ Language , 1993, IEEE Trans. Knowl. Data Eng..

[20]  Hong-Mei Chen Garcia,et al.  Multimedia Information Systems , 1996, Proceedings of HICSS-29: 29th Hawaii International Conference on System Sciences.

[21]  V. S. Subrahmanian,et al.  Multimedia Database Systems , 1993, Artificial Intelligence.

[22]  V. S. Subrahmanian,et al.  Hybrid knowledge bases for real-time robotic reasoning1 , 1994 .

[23]  Won Kim,et al.  Multimedia Information Management in an Object-Oriented Database System , 1987, VLDB.

[24]  Arding Hsu,et al.  Image processing on compressed data for large video databases , 1993, MULTIMEDIA '93.

[25]  Gio Wiederhold,et al.  Intelligent integration of information , 1993, SIGMOD Conference.