Synchronized Hypermedia Documents: a Model and its Applications

Authoring, delivering and presenting hypermedia documents is a complex task, since such documents can contain a large number of components bound by multiple temporal and layout constraints. Media items are files which have to be arranged on the screen and synchronized to be displayed to the user. The growth of network bandwidth encourages the use of rich media types in documents, such as animations, audio and video, giving new possibilities to improve information presentation. A distributed environment adds further complexity: documents can contain links to other documents, and the user can interact with components inside them by pausing, stopping or resuming their playback, or by following links to other documents. In this thesis we discuss synchronization issues in hypermedia presentations composed of several continuous and noncontinuous media objects. We define a model for synchronization relationships among media objects, and formally describe a presentation behavior in terms of events which help an automaton to evolve. We have also implemented a visual authoring environment based on the model, and suggested a number of applications in areas related to automatic generation of standard presentations and multimedia information retrieval.

[1]  Hugh C. Davis,et al.  Why use HyTime? , 1994, Electron. Publ..

[2]  Lloyd Rutledge,et al.  Do you have the time? Composition and linking in time-based hypermedia , 1999, Hypertext.

[3]  Lloyd Rutledge,et al.  Evaluating SMIL: three user case studies , 1999, MULTIMEDIA '99.

[4]  Rogério Ferreira Rodrigues,et al.  Modeling, authoring and formatting hypermedia documents in the HyperProp system , 2000, Multimedia Systems.

[5]  Steven R. Newcomb,et al.  The “HyTime ”: hypermedia/time-based document structuring language , 1991, CACM.

[6]  Gultekin Özsoyoglu,et al.  Query Processing Techniques for Multimedia Presentations , 2004, Multimedia Tools and Applications.

[7]  Ombretta Gaggi,et al.  Schema modelling for automatic generation of multimedia presentations , 2002, SEKE '02.

[8]  Ombretta Gaggi,et al.  Authoring and navigating hypermedia documents on the WWW , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[9]  Wendy T. Lucas,et al.  A visual approach to multimedia querying and presentation , 1997, MULTIMEDIA '97.

[10]  Elisa Bertino,et al.  Temporal Synchronization Models for Multimedia Data , 1998, IEEE Trans. Knowl. Data Eng..

[11]  Chung-Ming Huang,et al.  MING-I: a distributed interactive multimedia document development mechanism , 1998, Multimedia Systems.

[12]  Susanne Boll,et al.  ZYX - A Semantic Model for Multimedia Documents and Presentations , 1999, DS-8.

[13]  Howard Bowman,et al.  Synchronization in Multimedia Documents , 1998, EP.

[14]  Jin Yu A Simple, Intuitive Hypermedia Synchronization Model and its Realization in the Browser/Java Environment , 1997 .

[15]  Yves Chiaramella,et al.  Browsing and Querying: Two Complementary Approaches for Multimedia Information Retrieval , 1997, Hypertext, Information Retrieval, Multimedia.

[16]  Elke A. Rundensteiner,et al.  A Visual Multimedia Query Language for Temporal Analysis of Video Data , 1996, Multimedia Database Systems.

[17]  Nabil Layaïda,et al.  Madeus: an authoring environment for interactive multimedia documents , 1997, Proceedings of IEEE International Conference on Multimedia Computing and Systems.

[18]  Pietro Marchisio,et al.  MediaTouch: A Native Authoring Tool for MHEG-5 Applications , 2004, Multimedia Tools and Applications.

[19]  Gultekin Özsoyoglu,et al.  Querying Multimedia Presentations Based on Content , 1999, IEEE Trans. Knowl. Data Eng..

[20]  Dick C. A. Bulterman,et al.  Using the Amsterdam Hypermedia Model for Abstracting Presentation Behavior , 1995 .

[21]  Lynda Hardman,et al.  Modelling and authoring hypermedia documents , 1998 .

[22]  Elke A. Rundensteiner,et al.  User interface evaluation of a direct manipulation temporal visual query language , 1997, MULTIMEDIA '97.

[23]  K. Selçuk Candan,et al.  Retrieval schedules based on resource availability and flexible presentation specifications , 1998, Multimedia Systems.

[24]  Muriel Jourdan,et al.  A Scalable Toolkit for Designing Multimedia Authoring Environments , 2000, Multimedia Tools and Applications.

[25]  David A. Duce,et al.  A standard model for multimedia synchronization: PREMO synchronization objects , 1998, Multimedia Systems.

[26]  David Hung-Chang Du,et al.  Doing FLIPS: Flexible Interactive Presentation Synchronization , 1996, IEEE J. Sel. Areas Commun..

[27]  Arjeh M. Cohen,et al.  Synchronized Multimedia Integration Language (SMIL) 2.0 , 1998 .

[28]  Lloyd Rutledge,et al.  Generating presentation constraints from rhetorical structure , 2000, HYPERTEXT '00.

[29]  Ombretta Gaggi,et al.  Multimedia Reporting: Building Multimedia Presentations with Query Answers , 2001, Multimedia Information Systems.

[30]  Mayer D. Schwartz,et al.  The Dexter Hypertext Reference Model , 1994, CACM.

[31]  Renée J. Miller,et al.  Querying multimedia presentations , 1997, Proceedings of International Conference on Protocols for Multimedia Systems - Multimedia Networking.

[32]  V. S. Subrahmanian,et al.  Multimedia Database Systems , 1993, Artificial Intelligence.

[33]  Henning Schulzrinne,et al.  Real Time Streaming Protocol (RTSP) , 1998, RFC.

[34]  Nabil Layaïda,et al.  Madeus, and authoring environment for interactive multimedia documents , 1998, MULTIMEDIA '98.

[35]  Ombretta Gaggi,et al.  Retrieving Consistent Multimedia Presentation Fragments , 2002, Multimedia Information Systems.

[36]  Ombretta Gaggi,et al.  Querying and Browsing Multimedia Presentations , 2001, MDIC.

[37]  Lloyd Rutledge,et al.  GRiNS: A GRaphical INterface for Creating and Playing SMIL Documents , 1998, Comput. Networks.

[38]  Dick C. A. Bulterman,et al.  CMIFed: a presentation environment for portable hypermedia documents , 1993, MULTIMEDIA '93.

[39]  Chung-Ming Huang,et al.  Synchronization for Interactive Multimedia Presentations , 1998, IEEE Multim..

[40]  David H. C. Du,et al.  Building A Framework for FLexible Interactive Presentations , 1996 .

[41]  Alberto Del Bimbo,et al.  Visual information retrieval , 1999 .

[42]  Nils Klarlund,et al.  DSD: A schema language for XML , 2000, FMSP '00.

[43]  Susanne Boll,et al.  ZYX - A Multimedia Document Model for Reuse and Adaption , 2001 .

[44]  Thomas Rist,et al.  Relating the primitive hierarchy of the PREMO standard to the standard reference model for intelligent multimedia presentation systems , 1997, Comput. Stand. Interfaces.

[45]  V. S. Subrahmanian,et al.  An algebra for creating and querying multimedia presentations , 2000, Multimedia Systems.

[46]  K. Selçuk Candan,et al.  CHIMP: a framework for supporting distributed multimedia document authoring and presentation , 1997, MULTIMEDIA '96.

[47]  Maria Cristina Ferreira de Oliveira,et al.  Hypercharts: Extended Statecharts to Support Hypermedia Specification , 1999, IEEE Trans. Software Eng..

[48]  Renée J. Miller,et al.  Integrating Hierarchical Navigation and Querying: A User Customizable Solution , 1995 .

[49]  Ombretta Gaggi,et al.  A synchronization model for hypermedia documents navigation , 2000, SAC '00.

[50]  Lloyd Rutledge,et al.  Towards second and third generation web-based multimedia , 2001, WWW '01.

[51]  James F. Allen Maintaining knowledge about temporal intervals , 1983, CACM.

[52]  Chunlei Liu,et al.  Multimedia over IP: RSVP, RTP, RTCP, RTSP , 1999 .

[53]  Tran Cao Son,et al.  A Multimedia display extension to SQL: Language Design and Architecture , 1997, ICDE 1997.

[54]  Lynda Hardman,et al.  Application-Specic Constraints for Multimedia Presentation Generation , 2001 .

[55]  Chitta Baral,et al.  SQL+D: extended display capabilities for multimedia database queries , 1998, MULTIMEDIA '98.

[56]  Timos K. Sellis,et al.  Spatio-temporal composition and indexing for large multimedia applications , 1998, Multimedia Systems.

[57]  Gregory D. Abowd,et al.  Automated capture, integration, and visualization of multiple media streams , 1998, Proceedings. IEEE International Conference on Multimedia Computing and Systems (Cat. No.98TB100241).