Designing annotation before it's needed

This paper considers the automated and semi-automated annotation of audiovisual media in a new type of production framework, A4SM (Authoring System for Syntactic, Semantic and Semiotic Modelling). We present the architecture of the framework and outline the underlying XML-Schema based content description structures of A4SM. We then describe tools for a news and demonstrate how video material can be annotated in real time and how this information can not only be used for retrieval but also can be used during the different phases of the production process itself. Finally, we discuss the pros and cons of our approach of evolving semantic networks as the basis for audio- visual content description.

[1]  P. Bloom,et al.  High-quality digital audio in the entertainment industry: An overview of achievements and challenges , 1985, IEEE ASSP Magazine.

[2]  C. Hartshorne,et al.  Collected Papers of Charles Sanders Peirce , 1935, Nature.

[3]  S. S. Rath,et al.  Conference proceedings , 1999, 1987 IEEE Applied Power Electronics conference and Exposition.

[4]  Glorianna Davenport,et al.  The Stratification System - A Design Emvironment for Random Access , 1992, NOSSDAV.

[5]  Yihong Gong,et al.  Automatic parsing of news video , 1994, 1994 Proceedings of IEEE International Conference on Multimedia Computing and Systems.

[6]  T. Mika,et al.  HYBRIDES TRACKINGSYSTEM FUR VIRTUELLE STUDIOS , 1999 .

[7]  François Pachet,et al.  A taxonomy of musical genres , 2000, RIAO.

[8]  李幼升,et al.  Ph , 1989 .

[9]  Max Mühlhäuser,et al.  Design Patterns for Interactive Musical Systems , 1998, IEEE Multim..

[10]  Karen Spärck Jones,et al.  Audio Indexing and Retrieval of Complete Broadcoast News Shows , 2000, RIAO.

[11]  G. Halasz Frank,et al.  Reflections on NoteCards: seven issues for the next generation of hypermedia systems , 1988 .

[12]  Alan P. Parkes.  An artificial intelligence approach to the conceptual description of videodisc images , 1988 .

[13]  Frank Nack,et al.  Everything You Wanted to Know About MPEG-7: Part 1 , 1999, IEEE Multim..

[14]  Jane Hunter,et al.  Combining RDF and XML schemas to enhance interoperability between metadata application profiles , 2001, WWW '01.

[15]  FRANK NACK,et al.  Toward the Automated Editing of Theme Oriented Video Sequences , 1997, Appl. Artif. Intell..

[16]  Jane Hunter,et al.  A Comparison of Schemas for Video Metadata Representation , 1999, Comput. Networks.

[17]  Douglas Keislar,et al.  Content-Based Classification, Search, and Retrieval of Audio , 1996, IEEE Multim..

[18]  Craig A. Lindley.  A Video Annoation Methodology for Interactive Video Sequence Generation , 2001, Digital Content Creation.

[19]  R. Arnheim.  Art and Visual Perception, a Psychology of the Creative Eye , 1967 .

[20]  Keiji Hirata.  Towards Formalizing Jazz Piano Knowledge with a Deductive Object-Oriented Approach , 1995 .

[21]  David Pye,et al.  AT_TV: Broadcast Television and Radio Retrieval , 2000, RIAO.

[22]  Alan P. Parkes.  Settings and the setting structure: the description and automated propagation of networks for perusing videodisk image states , 1989, SIGIR '89.

[23]  R. Arnheim,et al.  Art and Visual Perception: A Psychology of the Creative Eye. , 1956 .

[24]  Umberto Eco,et al.  A theory of semiotics , 1976, Advances in semiotics.

[25]  Judy Robertson,et al.  Real-time music generation for a virtual environment , 1998 .

[26]  Nicola Orio,et al.  SMILE: a System for Content-based Musical Information Retrieval Environments , 2000, RIAO.

[27]  P. Beek,et al.  Text of 15938-5 FCD Information Technology-Multimedia Content Description Interface-Pard 5 Multimedia Description Schemes , 2001 .

[28]  Yukinobu Taniguchi,et al.  Structured Video Computing , 1994, IEEE MultiMedia.

[29]  Boon-Lock Yeo,et al.  Video browsing using clustering and scene transitions on compressed sequences , 1995, Electronic Imaging.

[30]  Alberto Del Bimbo,et al.  Visual information retrieval , 1999 .

[31]  Wolfgang Effelsberg,et al.  Automatic audio content analysis , 1997, MULTIMEDIA '96.

[32]  Kevin Michael Brooks.  Metalinear cinematic narrative : theory, process, and tool , 1999 .

[33]  Simone Santini,et al.  Integrated browsing and querying for image databases , 2000, IEEE MultiMedia.

[34]  Frank G. Halasz,et al.  Reflections on NoteCards: seven issues for the next generation of hypermedia systems , 1987, CACM.

[35]  Jorma Tarhio,et al.  Searching monophonic patterns within polyphonic sources , 2000 .

[36]  Philippe Aigrain,et al.  Medium knowledge-based macro-segmentation of video into sequences , 1997 .

[37]  John F. Sowa,et al.  Conceptual Structures: Information Processing in Mind and Machine , 1983 .

[38]  Marc Davis,et al.  Media streams: representing video for retrieval and repurposing , 1994, MULTIMEDIA '94.

[39]  Akio Nagasaka,et al.  Automatic Video Indexing and Full-Video Search for Object Appearances , 1991, VDB.

[40]  Craig Lindley,et al.  Environments for the production and maintenance of interactive stories , 2000 .

[41]  Alberto Del Bimbo,et al.  Semantics in Visual Information Retrieval , 1999, IEEE Multim..

[42]  Shih-Fu Chang,et al.  Overview of the MPEG-7 standard , 2001, IEEE Trans. Circuits Syst. Video Technol..