Media Streams: an iconic visual language for video annotation

In order to enable the search and retrieval of video from large archives, we need a representation of video content. Although some aspects of video can be automatically parsed, a detailed representation requires that video be annotated. We discuss the design criteria for a video annotation language with special attention to the issue of creating a global, reusable video archive. Our prototype system, Media Streams, enables users to create multi-layered, iconic annotations of streams video data. Within Media Streams, the organization and categories of the Director's Workshop allow users to browse and compound over 2200 iconic primitives by means of a cascading hierarchical structure which supports compounding icons across branches of the hierarchy. The problems of creating a representation of action for video are given special attention, as well as describing transitions in video.<<ETX>>

[1]  Gennaro Costagliola,et al.  A methodology for iconic language design with application to augmentative communication , 1992, Proceedings IEEE Workshop on Visual Languages.

[2]  Ron MacNeil.  Generating multimedia presentations automatically using TYRO, the constraint, case-based designer's apprentice , 1991, Proceedings 1991 IEEE Workshop on Visual Languages.

[3]  S. Eisenstein,et al.  The Film Sense , 1942 .

[4]  Michael Hawley.  Structure out of sound , 1993 .

[5]  Lev Kuleshov,et al.  Kuleshov on Film: Writings by Lev Kuleshov , 1975 .

[6]  Yoshinobu Tonomura,et al.  Video browsing using brightness data , 1991, Other Conferences.

[7]  Shinji Abe,et al.  Content oriented visual interface using video icons for visual database systems , 1990, J. Vis. Lang. Comput..

[8]  Robert R. Korfhage,et al.  Criteria for Iconic Languages , 1986 .

[9]  David Bordwell,et al.  Film Art: An Introduction , 1979 .

[10]  Robert R. Korfhage,et al.  Features and a model for icon morphological transformation , 1991, Proceedings 1991 IEEE Workshop on Visual Languages.

[11]  Kenneth B. Haase,et al.  FRAMER: A Persistent Portable Representation Library , 1994, ECAI.

[12]  Barry Arons,et al.  SpeechSkimmer: interactively skimming recorded speech , 1993, UIST '93.

[13]  Gilles R. Bloch.  From Concepts to Film Sequences , 1988, RIAO.

[14]  Shi-Kuo Chang.  Introduction: Visual Languages and Iconic Languages , 1986 .

[15]  Alberto Del Bimbo,et al.  A spatio-temporal logic for image sequence coding and retrieval , 1992, Proceedings IEEE Workshop on Visual Languages.

[16]  S. Eisenstein,et al.  Film Form: Essays in Film Theory , 1949 .

[17]  Henry Dreyfuss.  Symbol Sourcebook: An Authoritative Guide to International Graphic Symbols , 1972 .

[18]  Ramanathan V. Guha,et al.  Building Large Knowledge-Based Systems: Representation and Inference in the Cyc Project , 1990 .

[19]  John Preston Isenhour.  The effects of context and order in film editing , 1975 .

[20]  Ephraim P. Glinert,et al.  Visual tools and languages: directions for the '90s , 1991, Proceedings 1991 IEEE Workshop on Visual Languages.

[21]  David Bordwell.  Narration in the Fiction Film , 1985 .

[22]  Alberto Del Bimbo,et al.  Sequence retrieval by contents through spatio temporal indexing , 1993, Proceedings 1993 IEEE Symposium on Visual Languages.

[23]  Michael Mills,et al.  A magnifier tool for video data , 1992, CHI.

[24]  Akio Nagasaka,et al.  Automatic structure visualization for video editing , 1993, INTERCHI.

[25]  斉藤 康己,et al.  Douglas B. Lenat and R. V. Guha : Building Large Knowledge-Based Systems, Representation and Inference in the Cyc Project, Addison-Wesley (1990). , 1990 .

[26]  Noël Burch.  Theory of film practice , 1973 .

[27]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[28]  Edward Lee Elliott.  Watch-grab-arrange-see : thinking with motion images via streams and collages , 1993 .

[29]  Takafumi Miyatake,et al.  IMPACT: an interactive natural-motion-picture dedicated multimedia authoring system , 1991, CHI.

[30]  Shi-Kuo Chang,et al.  Image sequence compression by iconic indexing , 1989, [Proceedings] 1989 IEEE Workshop on Visual Languages.