Detection of Stochastic Processes

This paper reviews two streams of development, from the 1940's to the present, in signal detection theory: the structure of the likelihood ratio for detecting signals in noise and the role of dynamic optimization in detection problems involving either very large signal sets or the joint optimization of observation time and performance. This treatment deals exclusively with basic results developed for the situation in which the observations are modeled as continuous-time stochastic processes. The mathematics and intuition behind such developments as the matched filter, the RAKE receiver, the estimator-correlator, maximum-likelihood sequence detectors, multiuser detectors, sequential probability ratio tests, and cumulative-sum quickest detectors, are described.

[1]  K. F. Gauss,et al.  Theoria combinationis observationum erroribus minimis obnoxiae , 1823 .

[2]  R. Fisher,et al.  On the Mathematical Foundations of Theoretical Statistics , 1922 .

[3]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[4]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[5]  W. T. Martin,et al.  Transformations of Weiner Integrals Under Translations , 1944 .

[6]  J. V. Vleck,et al.  A Theoretical Comparison of the Visual, Aural, and Meter Reception of Pulsed Signals in the Presence of Noise , 1946 .

[7]  Dennis Gabor,et al.  Theory of communication , 1946 .

[8]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series, with Engineering Applications , 1949 .

[9]  U. Grenander.  Stochastic processes and statistical inference , 1950 .

[10]  H. W. Bode,et al.  A Simplified Derivation of Linear Least Square Smoothing and Prediction Theory , 1950, Proceedings of the IRE.

[11]  L. Zadeh,et al.  An Extension of Wiener's Theory of Prediction , 1950 .

[12]  Abraham Wald,et al.  Statistical Decision Functions , 1951 .

[13]  Kiyosi Itô.  Stochastic Differential Equations , 2018, The Control Systems Handbook.

[14]  J. Kiefer,et al.  Sequential Decision Problems for Processes with Continuous time Parameter. Testing Hypotheses , 1953 .

[15]  J. Doob.  Stochastic processes , 1953 .

[16]  W. W. Peterson,et al.  The theory of signal detectability , 1954, Trans. IRE Prof. Group Inf. Theory.

[17]  Philip M. Woodward,et al.  Probability and Information Theory with Applications to Radar , 1954 .

[18]  Robert Price,et al.  Optimum detection of random signals in noise, with application to scatter-multipath communication-I , 1956, IRE Trans. Inf. Theory.

[19]  William M. Siebert,et al.  A radar detection philosophy , 1956, IRE Trans. Inf. Theory.

[20]  Robert E. Kalaba,et al.  On the role of dynamic programming in statistical communication theory , 1957, IRE Trans. Inf. Theory.

[21]  P. E. Green,et al.  A Communication Technique for Multipath Channels , 1958, Proceedings of the IRE.

[22]  George L. Turin,et al.  The theory of optimum noise immunity , 1959 .

[23]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[24]  R. L. Stratonovich.  CONDITIONAL MARKOV PROCESSES , 1960 .

[25]  I. V. Girsanov.  On Transforming a Certain Class of Stochastic Processes by Absolutely Continuous Substitution of Measures , 1960 .

[26]  G. Turin,et al.  An introduction to matched filters , 1960, IRE Trans. Inf. Theory.

[27]  R. E. Kalman,et al.  New Results in Linear Filtering and Prediction Theory , 1961 .

[28]  E. Parzen.  An Approach to Time Series Analysis , 1961 .

[29]  E. Parzen.  Extraction and Detection Problems and Reproducing Kernel Hilbert Spaces , 1962 .

[30]  J. Hájek.  On linear statistical problems in stochastic processes , 1962 .

[31]  A. Shiryaev.  On Optimum Methods in Quickest Detection Problems , 1963 .

[32]  D. O. North,et al.  An Analysis of the factors which determine signal/noise discrimination in pulsed-carrier systems , 1963 .

[33]  M. Rosenblatt,et al.  SINGULAR GAUSSIAN MEASURES IN DETECTION THEORY , 1963 .

[34]  H. Kushner.  On the Differential Equations Satisfied by Conditional Probablitity Densities of Markov Processes, with Applications , 1964 .

[35]  I. M. Jacobs,et al.  Principles of Communication Engineering , 1965 .

[36]  E. Wong,et al.  ON THE RELATION BETWEEN ORDINARY AND STOCHASTIC DIFFERENTIAL EQUATIONS , 1965 .

[37]  Fred C. Schweppe,et al.  Evaluation of likelihood functions for Gaussian signals , 1965, IEEE Trans. Inf. Theory.

[38]  R. L. Stratonovich.  A New Representation for Stochastic Integrals and Equations , 1966 .

[39]  R. Bellman.  Dynamic programming. , 1957, Science.

[40]  L. Shepp.  Radon-Nikodym Derivatives of Gaussian Measures , 1966 .

[41]  Thomas Kailath,et al.  A projection method for signal detection in colored Gaussian noise , 1967, IEEE Trans. Inf. Theory.

[42]  T. T. Kadota,et al.  Differentiation of Karhunen-Loève expansion and application to optimum reception of sure signals in noise , 1967, IEEE Trans. Inf. Theory.

[43]  H. Kunita,et al.  On Square Integrable Martingales , 1967, Nagoya Mathematical Journal.

[44]  Van Nostrand,et al.  Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm , 1967 .

[45]  T. Kailath,et al.  An innovations approach to least-squares estimation--Part II: Linear smoothing in additive white noise , 1968 .

[46]  Tyrone E. Duncan,et al.  Evaluation of Likelihood Functions , 1968, Inf. Control..

[47]  Harry L. Van Trees,et al.  Detection, Estimation, and Modulation Theory, Part I , 1968 .

[48]  C. Helstrom,et al.  Statistical theory of signal detection , 1968 .

[49]  M. Zakai.  On the optimal filtering of diffusion processes , 1969 .

[50]  J. Andel.  Sequential Analysis , 2022, The SAGE Encyclopedia of Research Design.

[51]  C. Helstrom.  Quantum detection and estimation theory , 1969 .

[52]  Thomas Kailath,et al.  A general likelihood-ratio formula for random signals in Gaussian noise , 1969, IEEE Trans. Inf. Theory.

[53]  J. Omura,et al.  On the Viterbi decoding algorithm , 1969, IEEE Trans. Inf. Theory.

[54]  J. Thomas.  Nonparametric detection , 1970 .

[55]  H. Akaike,et al.  Comment on "An innovations approach to least-squares estimation, part I: Linear filtering in additive white noise" , 1970 .

[56]  Thomas Kailath,et al.  Likelihood ratios for Gaussian processes , 1970, IEEE Trans. Inf. Theory.

[57]  G. Wahba,et al.  Some results on Tchebycheffian spline functions , 1971 .

[58]  Thomas Kailath,et al.  RKHS approach to detection and estimation problems-I: Deterministic signals in Gaussian noise , 1971, IEEE Trans. Inf. Theory.

[59]  P. Meyer,et al.  Séminaire de probabilités X, Université de Strasbourg , 1971 .

[60]  G. Lorden.  PROCEDURES FOR REACTING TO A CHANGE IN DISTRIBUTION , 1971 .

[61]  T. Kailath.  The Structure of Radon-Nikodym Derivatives with Respect to Wiener and Related Measures , 1971 .

[62]  G. David Forney,et al.  Maximum-likelihood sequence estimation of digital sequences in the presence of intersymbol interference , 1972, IEEE Trans. Inf. Theory.

[63]  Thomas Kailath,et al.  An RKHS approach to detection and estimation problems- III: Generalized innovations representations and a likelihood-ratio formula , 1972, IEEE Trans. Inf. Theory.

[64]  Thomas Kailath,et al.  Some relations among RKHS norms, Fredholm equations, and innovations representations , 1972, IEEE Trans. Inf. Theory.

[65]  D. Messerschmitt.  A Geometric Theory of Intersymbol Interference , 1973 .

[66]  Thomas Kailath,et al.  RKHS approach to detection and estimation problems-IV: Non-Gaussian detection , 1973, IEEE Trans. Inf. Theory.

[67]  Thomas Kailath,et al.  RKHS approach to detection and estimation problems-V: Parameter estimation , 1973, IEEE Trans. Inf. Theory.

[68]  T. Kailath,et al.  Radon-Nikodym Derivatives with Respect to Measures Induced by Discontinuous Independent-Increment Processes , 1975 .

[69]  Thomas Kailath,et al.  An RKHS approach to detection and estimation problems-II: Gaussian signal detection , 1975, IEEE Trans. Inf. Theory.

[70]  Donald L. Snyder,et al.  Random point processes , 1975 .

[71]  Thomas Kailath,et al.  The modeling of randomly modulated jump processes , 1975, IEEE Trans. Inf. Theory.

[72]  G. Wahba.  Interpolating Spline Methods for Density Estimation I. Equi-Spaced Knots , 1975 .

[73]  Sándor Csibi,et al.  Stochastic processes with learning properties , 1975 .

[74]  Malcolm H. Davis,et al.  Exact and approximate filtering in signal detection - An example , 1977 .

[75]  J. Lehoczky.  FORMULAS FOR STOPPED DIFFUSION PROCESSES WITH STOPPING TIMES BASED ON THE MAXIMUM , 1977 .

[76]  Mark H. A. Davis,et al.  Exact and approximate filtering in signal detection: An example (Corresp.) , 1977, IEEE Trans. Inf. Theory.

[77]  J. Spiliotis.  Sur les intégrales stochastiques de L. C. Young , 1979 .

[78]  H. Weinert,et al.  Vector-valued Lg-splines I. Interpolating splines , 1979 .

[79]  Ralph Otto Schmidt,et al.  A signal subspace approach to multiple emitter location and spectral estimation , 1981 .

[80]  P. Brémaud.  Point Processes and Queues , 1981 .

[81]  H. Weinert.  Reproducing kernel Hilbert spaces: Applications in statistical signal processing , 1982 .

[82]  P. Brémaud.  Point processes and queues, martingale dynamics , 1983 .

[83]  H. Vincent Poor,et al.  Robust matched filters , 1983, IEEE Trans. Inf. Theory.

[84]  E. Wong,et al.  Stochastic Processes in Engineering Systems , 1984 .

[85]  C.E. Shannon,et al.  Communication in the Presence of Noise , 1949, Proceedings of the IRE.

[86]  S.A. Kassam,et al.  Robust techniques for signal processing: A survey , 1985, Proceedings of the IEEE.

[87]  Sergio Verdú,et al.  Minimum probability of error for asynchronous Gaussian multiple-access channels , 1986, IEEE Trans. Inf. Theory.

[88]  G. Moustakides.  Optimal stopping times for detecting changes in distributions , 1986 .

[89]  S. Verdú,et al.  Abstract dynamic programming models under commutativity conditions , 1987 .

[90]  S. Stigler,et al.  The History of Statistics: The Measurement of Uncertainty before 1900 , 1986 .

[91]  S. Kassam.  Nonparametric Hard Limiting and Sign Detection of Narrow-Band Deterministic and Random Signals , 1987 .

[92]  Ioannis Karatzas,et al.  Brownian Motion and Stochastic Calculus , 1987 .

[93]  Sergio Verdú,et al.  Maximum likelihood sequence detection for intersymbol interference channels: A new upper bound on error probability , 1987, IEEE Trans. Inf. Theory.

[94]  S. Kassam.  Signal Detection in Non-Gaussian Noise , 1987 .

[95]  H. Vincent Poor,et al.  Signal detection in fractional Gaussian noise , 1988, IEEE Trans. Inf. Theory.

[96]  A. F. Gualtierotti,et al.  Likelihood-Ratio Detection of Stochastic Signals. , 1989 .

[97]  Thomas Kailath,et al.  ESPRIT-estimation of signal parameters via rotational invariance techniques , 1989, IEEE Trans. Acoust. Speech Signal Process..

[98]  Sergio Verdú,et al.  Linear multiuser detectors for synchronous code-division multiple-access channels , 1989, IEEE Trans. Inf. Theory.

[99]  M. Yor,et al.  Continuous martingales and Brownian motion , 1990 .

[100]  H. Vincent Poor,et al.  An RKHS approach to robust L2 estimation and signal detection , 1990, IEEE Trans. Inf. Theory.

[101]  Craig K. Rushforth,et al.  A Family of Suboptimum Detectors for Coherent Multiuser Communications , 1990, IEEE J. Sel. Areas Commun..

[102]  Donald L. Snyder,et al.  Random Point Processes in Time and Space , 1991 .

[103]  T. Bayes.  An essay towards solving a problem in the doctrine of chances , 2003 .

[104]  H. Vincent Poor,et al.  On generalized signal-to-noise ratios in quadratic detection , 1992, Math. Control. Signals Syst..

[105]  J. Tsitsiklis.  Decentralized Detection' , 1993 .

[106]  C. L. Nikias,et al.  Higher-order spectra analysis : a nonlinear signal processing framework , 1993 .

[107]  Michèle Basseville,et al.  Detection of abrupt changes: theory and application , 1993 .

[108]  M. Beibel.  Bayes problems in change-point models for the Wiener process , 1994 .

[109]  Carl W. Helstrom,et al.  Elements of signal detection and estimation , 1994 .

[110]  M. Woodroofe,et al.  A Generalized Parking Problem , 1994 .

[111]  H. V. Poor,et al.  Detection of non-Gaussian signals: a paradigm for modern statistical signal processing , 1994, Proc. IEEE.

[112]  Michèle Basseville,et al.  Detection of Abrupt Changes: Theory and Applications. , 1995 .

[113]  Upamanyu Madhow,et al.  Blind adaptive multiuser detection , 1995, IEEE Trans. Inf. Theory.

[114]  Trieu-Kien Truong,et al.  Spectral representation of fractional Brownian motion in n dimensions and its properties , 1995, IEEE Trans. Inf. Theory.

[115]  Jerry D. Gibson,et al.  Introduction to Nonparametric Detection with Applications , 1995 .

[116]  A. Shiryaev.  COMMUNICATIONS OF THE MOSCOW MATHEMATICAL SOCIETY: Minimax optimality of the method of cumulative sums (cusum) in the case of continuous time , 1996 .

[117]  M. Beibel.  A note on Ritov's Bayes approach to the minimax property of the cusum procedure , 1996 .

[118]  Arogyaswami Paulraj,et al.  Space-time processing for wireless communications , 1997 .

[119]  Rick S. Blum,et al.  Distributed detection with multiple sensors I. Advanced topics , 1997, Proc. IEEE.

[120]  H. Vincent Poor,et al.  Probability of error in MMSE multiuser detection , 1997, IEEE Trans. Inf. Theory.

[121]  Pramod K. Varshney,et al.  Distributed detection with multiple sensors I. Fundamentals , 1997, Proc. IEEE.

[122]  H. Vincent Poor,et al.  Performance analysis of sequential tests between Poisson processes , 1997, IEEE Trans. Inf. Theory.

[123]  A.J. Paulraj,et al.  Space-time processing for wireless communications , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[124]  H. R. Lerche,et al.  A New Look at Optimal Stopping Problems related to Mathematical Finance , 1997 .

[125]  Jung-Fu Cheng,et al.  Turbo Decoding as an Instance of Pearl's "Belief Propagation" Algorithm , 1998, IEEE J. Sel. Areas Commun..

[126]  H. Vincent Poor,et al.  Blind Multiuser Detection: A Subspace Approach , 1998, IEEE Trans. Inf. Theory.

[127]  H. Vincent Poor,et al.  Wireless communications : signal processing perspectives , 1998 .

[128]  Sergio Verdu,et al.  Multiuser Detection , 1998 .

[129]  Bernard Fino,et al.  Multiuser detection: , 1999, Ann. des Télécommunications.