Smallest singular value and limit eigenvalue distribution of a class of non-Hermitian random matrices with statistical application

Suppose $X$ is an $N \times n$ complex matrix whose entries are centered, independent, and identically distributed random variables with variance $1/n$ and whose fourth moment is of order ${\mathcal O}(n^{-2})$. In the first part of the paper, we consider the non-Hermitian matrix $X A X^* - z$, where $A$ is a deterministic matrix whose smallest and largest singular values are bounded below and above respectively, and $z\neq 0$ is a complex number. Asymptotic probability bounds for the smallest singular value of this model are obtained in the large dimensional regime where $N$ and $n$ diverge to infinity at the same rate. In the second part of the paper, we consider the special case where $A = J = [1_{i-j = 1\mod n} ]$ is a circulant matrix. Using the result of the first part, it is shown that the limit eigenvalue distribution of $X J X^*$ exists in the large dimensional regime, and we determine this limit explicitly. A statistical application of this result devoted towards testing the presence of correlations within a multivariate time series is considered. Assuming that $X$ represents a ${\mathbb C}^N$-valued time series which is observed over a time window of length $n$, the matrix $X J X^*$ represents the one-step sample autocovariance matrix of this time series. Guided by the result on the limit spectral measure of this matrix, a whiteness test against an MA correlation model on the time series is introduced. Numerical simulations show the excellent performance of this test.