Wavelet-based statistical signal processing using hidden Markov models

Wavelet-based statistical signal processing techniques such as denoising and detection typically model the wavelet coefficients as independent or jointly Gaussian. These models are unrealistic for many real-world signals. We develop a new framework for statistical signal processing based on wavelet-domain hidden Markov models (HMMs) that concisely models the statistical dependencies and non-Gaussian statistics encountered in real-world signals. Wavelet-domain HMMs are designed with the intrinsic properties of the wavelet transform in mind and provide powerful, yet tractable, probabilistic signal models. Efficient expectation maximization algorithms are developed for fitting the HMMs to observational signal data. The new framework is suitable for a wide range of applications, including signal estimation, detection, classification, prediction, and even synthesis. To demonstrate the utility of wavelet-domain HMMs, we develop novel algorithms for signal denoising, classification, and detection.

[1]  H. Sorenson,et al.  Recursive bayesian estimation using gaussian sums , 1971 .

[2]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[3]  R. Redner,et al.  Mixture densities, maximum likelihood, and the EM algorithm , 1984 .

[4]  Edward J. Wegman,et al.  Statistical Signal Processing , 1985 .

[5]  R. Fildes.  Journal of the American Statistical Association : William S. Cleveland, Marylyn E. McGill and Robert McGill, The shape parameter for a two variable graph 83 (1988) 289-300 , 1989 .

[6]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[7]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[8]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[9]  L. Scharf,et al.  Statistical Signal Processing: Detection, Estimation, and Time Series Analysis , 1991 .

[10]  Neri Merhav,et al.  Universal classification for hidden Markov models , 1991, IEEE Trans. Inf. Theory.

[11]  Michèle Basseville,et al.  Modeling and estimation of multiresolution stochastic processes , 1992, IEEE Trans. Inf. Theory.

[12]  Patrick Flandrin,et al.  Wavelet analysis and synthesis of fractional Brownian motion , 1992, IEEE Trans. Inf. Theory.

[13]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[14]  Stéphane Mallat,et al.  Singularity detection and processing with wavelets , 1992, IEEE Trans. Inf. Theory.

[15]  Stéphane Mallat,et al.  Characterization of Signals from Multiscale Edges , 2011, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Jerome M. Shapiro,et al.  Embedded image coding using zerotrees of wavelet coefficients , 1993, IEEE Trans. Signal Process..

[17]  W. Clem Karl,et al.  Multiscale representations of Markov random fields , 1993, IEEE Trans. Signal Process..

[18]  John H. L. Hansen,et al.  Discrete-Time Processing of Speech Signals , 1993 .

[19]  Michèle Basseville,et al.  Detection of abrupt changes , 1993 .

[20]  Larry P. Heck,et al.  A multiscale stochastic modeling approach to the monitoring of mechanical systems , 1994, Proceedings of IEEE-SP International Symposium on Time- Frequency and Time-Scale Analysis.

[21]  Michael T. Orchard,et al.  An investigation of wavelet-based image coding using an entropy-constrained quantization framework , 1994, Proceedings of IEEE Data Compression Conference (DCC'94).

[22]  Charles A. Bouman,et al.  A multiscale random field model for Bayesian image segmentation , 1994, IEEE Trans. Image Process..

[23]  Ronald R. Coifman,et al.  Local discriminant bases , 1994, Optics & Photonics.

[24]  Jelena Kovacevic,et al.  Wavelets and Subband Coding , 2013, Prentice Hall Signal Processing Series.

[25]  J. R. Rohlicek,et al.  Parameter estimation of dependence tree models using the EM algorithm , 1995, IEEE Signal Processing Letters.

[26]  Yazhen Wang.  Jump and sharp cusp detection by wavelets , 1995 .

[27]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[28]  Richard Baraniuk,et al.  Hidden Markov models for wavelet-based signal processing , 1996, Conference Record of The Thirtieth Asilomar Conference on Signals, Systems and Computers.

[29]  H. Chipman,et al.  Signal de-noising using adaptive Bayesian wavelet shrinkage , 1996, Proceedings of Third International Symposium on Time-Frequency and Time-Scale Analysis (TFTS-96).

[30]  D. Roose,et al.  Bayesian Approach to Wavelet-based Image Processing , 1996 .

[31]  David Leporini,et al.  Bayesian approach to best basis selection , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[32]  Helmut Lucke,et al.  Which stochastic models allow Baum-Welch training? , 1996, IEEE Trans. Signal Process..

[33]  Edward H. Adelson,et al.  Noise removal via Bayesian wavelet coring , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[34]  E. Parzen,et al.  Data dependent wavelet thresholding in nonparametric regression with change-point applications , 1996 .

[35]  N. Lee,et al.  New method of linear time-frequency analysis for signal detection , 1996, Proceedings of Third International Symposium on Time-Frequency and Time-Scale Analysis (TFTS-96).

[36]  Michael T. Orchard,et al.  Image coding based on mixture modeling of wavelet coefficients and a fast estimation-quantization framework , 1997, Proceedings DCC '97. Data Compression Conference.

[37]  Eero P. Simoncelli,et al.  Progressive wavelet image coding based on a conditional probability model , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[38]  H. Chipman,et al.  Adaptive Bayesian Wavelet Shrinkage , 1997 .

[39]  Michael I. Jordan,et al.  Probabilistic Independence Networks for Hidden Markov Probability Models , 1997, Neural Computation.

[40]  B. Silverman,et al.  Wavelet thresholding via a Bayesian approach , 1998 .

[41]  An investigation of wavelet-based image coding using an entropy-constrained quantization framework , 1998, IEEE Trans. Signal Process..

[42]  R.G. Baraniuk,et al.  Simplified wavelet-domain hidden Markov models using contexts , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).