ICT Tools for Searching, Annotation and Analysis of Audiovisual Media

1. This report concerns the use of ICT tools in research in the arts and humanities using speech, mu­ sic, video and film in digital form, hereafter referred to as AV (audio­visual material). 2. The quantity of AV available to researchers is now massive and rapidly expanding, far exceeding the quantity of available print material in sheer number of bytes. 3. The main problem for researchers is no longer a paucity of AV but how to locate the material of in­ terest in the vast quantity available, and how to organise material once collected. 4. Metadata and tagging continue to be important to facilitate search. Standards for metadata for AV do exist but are not yet widely adopted. 5. Content­based search is becoming possible for speech, but is still beyond the horizon for music, and even more distant for video and film. Mixed speech, music and noise is very hard to search. 6. Copyright protection hampers research with AV, and digital rights management systems (DRM) threaten to prevent research altogether. 7. Once AV has been located and accessed, much research proceeds by annotation, for which many tools exist. Systems for reuse and sharing of annotations are in their infancy, however. 8. Many researchers make some kind of transcription of AV, and would value tools to automate this process. For speech, such tools exist with important limits to their accuracy and applicability. 9. Full music transcription tools do not exist, but researchers can benefit from other sorts of visualisa­ tions, for which tools do exist. 10. Researchers could work more effectively with better knowledge of ICT. A common failing is not so much ignorance of how to use particular tools as a misunderstanding of the processes the computer carries out and the validity of its results. 11. In Section 1.3, recommendations are made concerning: i. provision of ICT infrastructure for arts and humanities research, ii. training for researchers, iii. copyright law and digital rights management (DRM), iv. resource development unlikely to receive commercial support, v. dissemination of expertise and examples in research on AV with ICT, vi. standards and commercial tools, vii. metadata and digitisation projects outside the research community, viii. management of researchers' private collections of AV, ix. deposit and sharing of AV, including annotations of AV. and many others for informal conversations. We also gratefully acknowledge the generous amount of time and information given by all of the participants …