Learning to Tag using Noisy Labels

In order to organize and retrieve the ever growing collection of multimedia objects on the Web, many algorithms have been developed to automatically tag images, music and videos. One source of labeled data for training these algorithms are tags collected from the Web, via collaborative tagging websites (e.g., Flickr, Last.FM and YouTube) or crowdsourcing applications (e.g., human computation games and Mechanical Turk). A common approach is to use tags directly as labels for training algorithms in a supervised way. This approach is problematic, because the presence of synonyms and misspellings amongst the tags creates a label space that is overly fragmented, with a huge number of classes, many of which are sparse and semantically equivalent to one another. In this work, we investigate a method for training tagging algorithms using a reduced set of labels corresponding to topics derived from the tags. We show that our proposed method is comparable, in terms of annotation and retrieval performance, to the method of using tags directly as labels, while being more efficient to train (as there are fewer classes) and less wasteful (eliminating the need to discard tags that are associated with too few examples). We demonstrate our results using a dataset collected by a human computation game, called TagATune.