If It Sounds As Good As It Looks : Lessons Learned From Video Retrieval Evaluation

In many ways, music information retrieval (MIR) bears a closer resemblance to video information retrieval (VIR) than it does to text retrieval. Both music and video provide rich, complex sources of information having their own semantics. Both share the challenges of digitizing, segmenting and streaming, joined by problems relating to the representation of non-textual, non-verbal information. Because of this complexity, systems for the retrieval of these media pose unique challenges to evaluation including the construction of large testbeds, the crafting of representative topics for searching, and identification of appropriate metrics for evaluation. This paper will discuss recent efforts in video retrieval evaluation and how these efforts might inform the creation of an experimental evaluation environment for Music Information Retrieval.