MAVIS 2: a new approach to content and concept based navigation

One of the most active areas of multimedia research is into content based retrieval (CBR) of multimedia information. Using tools from the disparate disciplines of image processing, audio processing, video processing and others, approaches to CBR for the different media are being produced and integrated into multimedia systems, quite frequently in an ad hoc way. Also, traditional hypertext systems allow textual information to be arbitrarily linked so that users can navigate between related parts of the information in a system. Many of these systems can use multimedia information such as images, sounds and video clips that can also be linked. In such hypermedia systems the links are usually created using specific locations in particular documents. Some systems, such as Microcosm, also allow links to be created by specifying the text that forms one anchor of the link. These links are created once and can be followed from any location where the text occurs. To achieve this, the system has to examine documents currently being viewed by the user and look for matches between text in the documents and text that forms anchors of links within the system. Where matches occur the system can highlight the text as a source anchor for a link. This is a form of content based navigation(CBN). Navigational links are dynamically created by matching the content of currently viewed documents with the content of previously created link anchors. For textual documents this is relatively easy to implement since comparing text strings is often straightforward. Content based navigation for multimedia documents also involves comparing and matching selections of images, video and sound with each other. Our MAVIS 1 project addressed the problem of integrated content based retrieval and navigation from non-text media but there are many limitations associated with such systems. This paper will describe some of the reasons for these limitations and present an overview of MAVIS 2, a new architecture for multimedia content based retrieval and navigation. The architecture not only presents a consistent cross media approach to CBR and CBN but also includes the integration of a multimedia thesaurus which can substantially improve the flexibility and versatility of the multimedia information system, enhancing the capabilities of both content based retrieval and content based navigation. MAVIS stands for Multimedia Architecture for Video, Image and Sound and the MAVIS 2 architecture also includes integration of intelligent agents which support navigation by utilising both the media content and semantic concepts in the multimedia thesaurus to develop rapid paths from media based features to semantic concepts or to generate pseudo thesaurus groupings by feature clustering when no semantic relations are available. MAVIS 2 is currently being implemented and the paper will not only present details of the architecture but also show examples of retrieval and navigation with the prototype system.