Perturbation of linear forms of singular vectors under Gaussian noise

Let $A\in\mathbb{R}^{m\times n}$ be a matrix of rank $r$ with singular value decomposition (SVD) $A=\sum_{k=1}^r\sigma_k (u_k\otimes v_k),$ where $\{\sigma_k, k=1,\ldots,r\}$ are singular values of $A$ (arranged in a non-increasing order) and $u_k\in {\mathbb R}^m, v_k\in {\mathbb R}^n, k=1,\ldots, r$ are the corresponding left and right orthonormal singular vectors. Let $\tilde{A}=A+X$ be a noisy observation of $A,$ where $X\in\mathbb{R}^{m\times n}$ is a random matrix with i.i.d. Gaussian entries, $X_{ij}\sim\mathcal{N}(0,\tau^2),$ and consider its SVD $\tilde{A}=\sum_{k=1}^{m\wedge n}\tilde{\sigma}_k(\tilde{u}_k\otimes\tilde{v}_k)$ with singular values $\tilde{\sigma}_1\geq\ldots\geq\tilde{\sigma}_{m\wedge n}$ and singular vectors $\tilde{u}_k,\tilde{v}_k,k=1,\ldots, m\wedge n.$ The goal of this paper is to develop sharp concentration bounds for linear forms $\langle \tilde u_k,x\rangle, x\in {\mathbb R}^m$ and $\langle \tilde v_k,y\rangle, y\in {\mathbb R}^n$ of the perturbed (empirical) singular vectors in the case when the singular values of $A$ are distinct and, more generally, concentration bounds for bilinear forms of projection operators associated with SVD. In particular, the results imply upper bounds of the order $O\biggl(\sqrt{\frac{\log(m+n)}{m\vee n}}\biggr)$ (holding with a high probability) on $$\max_{1\leq i\leq m}\big|\big \big|\ \ {\rm and} \ \ \max_{1\leq j\leq n}\big|\big \big|,$$ where $b_k$ are properly chosen constants characterizing the bias of empirical singular vectors $\tilde u_k, \tilde v_k$ and $\{e_i^m,i=1,\ldots,m\}, \{e_j^n,j=1,\ldots,n\}$ are the canonical bases of $\mathbb{R}^m, {\mathbb R}^n,$ respectively.

[1]  V. Vu,et al.  Random perturbation of low rank matrices: Improving classical bounds , 2013, 1311.2657.

[2]  Mark Rudelson,et al.  Delocalization of eigenvectors of random matrices with independent entries , 2013, 1306.2887.

[3]  A. Rinaldo,et al.  Consistency of spectral clustering in stochastic block models , 2013, 1312.2050.

[4]  Van H. Vu,et al.  Random weighted projections, random quadratic forms and random eigenvectors , 2013, Random Struct. Algorithms.

[5]  Rongrong Wang,et al.  Singular Vector Perturbation Under Gaussian Noise , 2012, SIAM J. Matrix Anal. Appl..

[6]  Jiashun Jin,et al.  FAST COMMUNITY DETECTION BY SCORE , 2012, 1211.5803.

[7]  Santosh S. Vempala,et al.  Spectral Algorithms , 2009, Found. Trends Theor. Comput. Sci..

[8]  V. Koltchinskii,et al.  Asymptotics and Concentration Bounds for Spectral Projectors of Sample Covariance} \runtitle{Asymptotics and concentration of spectral projectors , 2014 .

[9]  Ling Huang,et al.  Spectral Clustering with Perturbed Data , 2008, NIPS.

[10]  Ren-Cang Li Relative Perturbation Theory: I. Eigenvalue and Singular Value Variations , 1998, SIAM J. Matrix Anal. Appl..

[11]  Van H. Vu Singular vectors under random perturbation , 2011, Random Struct. Algorithms.

[12]  Vladimir Koltchinskii,et al.  Asymptotics and Concentration Bounds for Bilinear Forms of Spectral Projectors of Sample Covariance , 2014, 1408.4643.

[13]  Raj Rao Nadakuditi,et al.  The singular values and vectors of low rank perturbations of large rectangular random matrices , 2011, J. Multivar. Anal..

[14]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[15]  Bin Yu,et al.  Spectral clustering and the high-dimensional stochastic blockmodel , 2010, 1007.1684.

[16]  G. Stewart Perturbation theory for the singular value decomposition , 1990 .