Joint multi-view representation and image annotation via optimal predictive subspace learning

Abstract Image representation and annotation are two key tasks in practical applications such as image search. Existing methods have tried to learn an effective representation or to predict tags directly using multi-view low-level visual features, which usually contain redundant information. However, these two tasks are closely related and interact on each other. A suitable image representation can yield better image annotation results, which in turn can effectively guide the image representation learning. In this paper, we propose to jointly conduct multi-view representation and image annotation via optimal predictive subspace learning, making the two tasks promote each other. Specifically, for subspace learning, visual structure and semantic information of images are exploited to make the learned subspace more discriminative and compact. For tag prediction, support vector machines (SVM) is adopted to obtain better tag prediction results. Then to simultaneously learn image representation, tag predictors and projection function, the three subproblems are combined into a unified optimization objective function and an alternative optimization algorithm is derived to solve it. Experimental results on four image datasets illustrate that our method is superior to the other image annotation methods.