Robust image annotation via simultaneous feature and sample outlier pursuit

Graph-based semi-supervised image annotation has achieved great success in a variety of studies, yet it essentially and intuitively suffers from both the irrelevant/noisy features (referred to as feature outliers) and the unusual/corrupted samples (referred to as sample outliers). In this work, we investigate how to derive robust sample affinity matrix via simultaneous feature and sample outlier pursuit. This task is formulated as a Dual-outlier and Prior-driven Low-Rank Representation (DP-LRR) problem, which possesses convexity in objective function. In DP-LRR, the clean data are assumed to be self-reconstructible with low-rank coefficient matrix as in LRR; while the error matrix is decomposed as the sum of a row-wise sparse matrix and a column-wise sparse matrix, the ℓ2,1-norm minimization of which encourages the pursuit of feature and sample outliers respectively. The DP-LRR is further regularized by the priors from side information, that is, the inhomogeneous data pairs. An efficient iterative procedure based on linearized alternating direction method is presented to solve the DP-LRR problem, with closed-form solutions within each iteration. The derived low-rank reconstruction coefficient matrix is then fed into any graph based semi-supervised label propagation algorithm for image annotation, and as a by-product, the cleaned data from DP-LRR can also be utilized as a better image representation to generally boost image annotation performance. Extensive experiments on MIRFlickr, Corel30K, NUS-WIDE-LITE and NUS-WIDE databases well demonstrate the effectiveness of the proposed formulation for robust image annotation.