Automatic Subspace Learning via Principal Coefficients Embedding

In this paper, we address two challenging problems in unsupervised subspace learning: 1) how to automatically identify the feature dimension of the learned subspace (i.e., automatic subspace learning) and 2) how to learn the underlying subspace in the presence of Gaussian noise (i.e., robust subspace learning). We show that these two problems can be simultaneously solved by proposing a new method [(called principal coefficients embedding (PCE)]. For a given data set <inline-formula> <tex-math notation="LaTeX">$\mathbf {D}\boldsymbol {\in } \mathbb{R}^{\boldsymbol {m\times n}}$ </tex-math></inline-formula>, PCE recovers a clean data set <inline-formula> <tex-math notation="LaTeX">$\mathbf {D}_{0}\boldsymbol {\in } \mathbb{R}^{\boldsymbol {m\times n}}$ </tex-math></inline-formula> from <inline-formula> <tex-math notation="LaTeX">$\mathbf {D}$ </tex-math></inline-formula> and simultaneously learns a global reconstruction relation <inline-formula> <tex-math notation="LaTeX">$\mathbf {C}\boldsymbol {\in } \mathbf {R}^{\boldsymbol {n\times n}}$ </tex-math></inline-formula> of <inline-formula> <tex-math notation="LaTeX">$\mathbf {D}_{\boldsymbol {0}}$ </tex-math></inline-formula>. By preserving <inline-formula> <tex-math notation="LaTeX">$\mathbf {C}$ </tex-math></inline-formula> into an <inline-formula> <tex-math notation="LaTeX">$\boldsymbol {m^{\prime }}$ </tex-math></inline-formula>-dimensional space, the proposed method obtains a projection matrix that can capture the latent manifold structure of <inline-formula> <tex-math notation="LaTeX">$\mathbf {D}_{\boldsymbol {0}}$ </tex-math></inline-formula>, where <inline-formula> <tex-math notation="LaTeX">$\boldsymbol {m^{\prime }\ll m}$ </tex-math></inline-formula> is automatically determined by the rank of <inline-formula> <tex-math notation="LaTeX">$\mathbf {C}$ </tex-math></inline-formula> with theoretical guarantees. PCE has three advantages: 1) it can automatically determine the feature dimension even though data are sampled from a union of multiple linear subspaces in presence of the Gaussian noise; 2) although the objective function of PCE only considers the Gaussian noise, experimental results show that it is robust to the non-Gaussian noise (e.g., random pixel corruption) and real disguises; and 3) our method has a closed-form solution and can be calculated very fast. Extensive experimental results show the superiority of PCE on a range of databases with respect to the classification accuracy, robustness, and efficiency.