Indexing, Browsing, and Searching of Digital Video and Digital Audio Information

In this chapter we examine various techniques for providing content access to information stored in a continuous medium, namely digital audio and digital video. Our coverage of audio is centered around post-processing the output of automatic recognition of speech or phones and we describe the various approaches that have been taken in this area. In order to give reasonable coverage of the possibilities and limitations of content-based access to digital video information we sketch out at a high level, the approaches taken in various video compression algorithms, principally the MPEG family.. We then address approaches to shot and scene boundary detection, choosing representative frames for browsing and for search, and various browsing interfaces that have been developed. We finish with an overview of the likely developments in this area in the future.