False Alarm Rate of the GLRT for Subspace Signals in Subspace Interference Plus Gaussian Noise

In the presence of interference and Gaussian noise with unknown covariance matrix, a generalized likelihood ratio test (GLRT) detector has already been developed for detecting a distributed target. The target signal and interference are described with subspace models, namely, the target signals (or interference) are modeled as linear combinations of the linearly independent columns of a known subspace matrix. In this paper, we obtain statistical properties of this GLRT detector, and prove its constant false alarm rate against the noise covariance matrix. Moreover, we derive analytical expressions for the probability of false alarm of the GLRT detector. Specifically, exact expressions for the probability of false alarm are obtained for six cases where the target subspace dimension (denoted by <inline-formula><tex-math notation="LaTeX">$p$</tex-math></inline-formula>) is 1, 2, and 3, and the number of range cells (denoted by <inline-formula><tex-math notation="LaTeX">$H$</tex-math></inline-formula>) the distributed target occupies is 1, 2, and 3. For the general case where <inline-formula><tex-math notation="LaTeX">$p\geq 4$</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">$H\geq 4$</tex-math></inline-formula>, we derive an approximate expression for the probability of false alarm. Monte Carlo simulations show that the exact expressions are valid, and the accuracy of the approximate expression is acceptable for moderate or large training data size. In practice, these expressions can greatly facilitate the threshold setting for the GLRT detector for any preassigned probability of false alarm.