Performance Analysis of Multi-Dimensional ESPRIT-Type Algorithms for Arbitrary and Strictly Non-Circular Sources With Spatial Smoothing

Spatial smoothing is a widely used preprocessing scheme to improve the performance of high-resolution parameter estimation algorithms in case of coherent signals or if only a small number of snapshots is available. In this paper, we present a first-order performance analysis of the spatially smoothed versions of <inline-formula> <tex-math notation="LaTeX">$R$</tex-math></inline-formula>-D Standard ESPRIT and <inline-formula> <tex-math notation="LaTeX">$R$</tex-math></inline-formula>-D Unitary ESPRIT for sources with arbitrary signal constellations as well as <inline-formula><tex-math notation="LaTeX">$R$</tex-math></inline-formula>-D NC Standard ESPRIT and <inline-formula><tex-math notation="LaTeX">$R$</tex-math></inline-formula>-D NC Unitary ESPRIT for strictly second-order (SO) non-circular (NC) sources. The derived expressions are asymptotic in the effective signal-to-noise ratio (SNR), i.e., the approximations become exact for either high SNRs or a large sample size. Moreover, no assumptions on the noise statistics are required apart from a zero mean and finite SO moments. We show that both <inline-formula><tex-math notation="LaTeX">$R$</tex-math></inline-formula>-D NC ESPRIT-type algorithms with spatial smoothing perform asymptotically identical in the high effective SNR regime. Generally, the performance of spatial smoothing based algorithms depends on the number of subarrays, which is a design parameter that needs to be chosen beforehand. In order to gain more insights into the optimal choice of the number of subarrays, we simplify the derived analytical <inline-formula><tex-math notation="LaTeX">$R$</tex-math></inline-formula>-D mean square error (MSE) expressions for the special case of a single source. The obtained MSE expression explicitly depends on the number of subarrays in each dimension, which allows us to analytically find the optimal number of subarrays for spatial smoothing. Based on this result, we additionally derive the maximum asymptotic gain from spatial smoothing and compute the asymptotic efficiency for the single source case in closed-form. All the analytical results are verified by simulations.