Capon algorithm mean-squared error threshold SNR prediction and probability of resolution

Below a specific threshold signal-to-noise ratio (SNR), the mean-squared error (MSE) performance of signal parameter estimates derived from the Capon algorithm degrades swiftly. Prediction of this threshold SNR point is of practical significance for robust system design and analysis. The exact pairwise error probabilities for the Capon (and Bartlett) algorithm, derived herein, are given by simple finite sums involving no numerical integration, include finite sample effects, and hold for an arbitrary colored data covariance. Via an adaptation of an interval error based method, these error probabilities, along with the local error MSE predictions of Vaidyanathan and Buckley, facilitate accurate prediction of the Capon threshold region MSE performance for an arbitrary number of well separated sources, circumventing the need for numerous Monte Carlo simulations. A large sample closed-form approximation for the Capon threshold SNR is provided for uniform linear arrays. A new, exact, two-point measure of the probability of resolution for the Capon algorithm, that includes the deleterious effects of signal model mismatch, is a serendipitous byproduct of this analysis that predicts the SNRs required for closely spaced sources to be mutually resolvable by the Capon algorithm. Last, a general strategy is provided for obtaining accurate MSE predictions that account for signal model mismatch.