On Approximate Maximum-Likelihood Methods for Blind Identification: How to Cope With the Curse of Dimensionality

We discuss approximate maximum-likelihood methods for blind identification and deconvolution. These algorithms are based on particle approximation versions of the expectation-maximization (EM) algorithm. We consider three different methods which differ in the way the posterior distribution of the symbols is computed. The first algorithm is a particle approximation method of the fixed-interval smoothing. The two-filter smoothing and the novel joined-two-filter smoothing involve an additional backward-information filter. Because the state space is finite, it is furthermore possible at each step to consider all the offsprings of any given particle. It is then required to construct a novel particle swarm by selecting, among all these offsprings, particle positions and computing appropriate weights. We propose here a novel unbiased selection scheme, which minimizes the expected loss with respect to general distance functions. We compare these smoothing algorithms and selection schemes in a Monte Carlo experiment. We show a significant performance increase compared to the expectation maximization Viterbi algorithm (EMVA), a fixed-lag smoothing algorithm and the Block constant modulus algorithm (CMA).

[1]  Michael Collins,et al.  Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms , 2002, EMNLP.

[2]  Eugene Charniak,et al.  Edge-Based Best-First Chart Parsing , 1998, VLC@COLING/ACL.

[3]  Robert D. Nowak,et al.  Compressed channel sensing , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[4]  Michael Isard,et al.  A Smoothing Filter for CONDENSATION , 1998, ECCV.

[5]  P. Loubaton,et al.  Blind cyclostationary statistics based carrier frequency offset and symbol timing delay estimators in flat-fading channels , 2001, 2001 MILCOM Proceedings Communications for Network-Centric Operations: Creating the Information Force (Cat. No.01CH37277).

[6]  Jill K. Nelson,et al.  Bayesian MLSD for multipath Rayleigh fading channels , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  J. Preisig,et al.  Estimation of Rapidly Time-Varying Sparse Channels , 2007, IEEE Journal of Oceanic Engineering.

[8]  P. Fearnhead,et al.  Improved particle filter for nonlinear problems , 1999 .

[9]  Paul A. Viola,et al.  Learning A* underestimates: Using inference to guide inference , 2007, AISTATS.

[10]  José A. R. Fonollosa,et al.  Blind channel estimation and data detection using hidden Markov models , 1997, IEEE Trans. Signal Process..

[11]  Georgios B. Giannakis,et al.  Basis expansion models and diversity techniques for blind identification and equalization of time-varying channels , 1998, Proc. IEEE.

[12]  D. Donoho For most large underdetermined systems of equations, the minimal 𝓁1‐norm near‐solution approximates the sparsest near‐solution , 2006 .

[13]  Stéphane Boucheron,et al.  Optimal error exponents in hidden Markov models order estimation , 2003, IEEE Trans. Inf. Theory.

[14]  A. Doucet,et al.  A survey of convergence results on particle ltering for practitioners , 2002 .

[15]  Steve J. Young,et al.  A One Pass Decoder Design For Large Vocabulary Recognition , 1994, HLT.

[16]  Octavia A. Dobre,et al.  Likelihood-Based Algorithms for Linear Digital Modulation Classification in Fading Channels , 2006, 2006 Canadian Conference on Electrical and Computer Engineering.

[17]  Nambi Seshadri,et al.  Joint data and channel estimation using blind trellis search techniques , 1994, IEEE Trans. Commun..

[18]  Patrick Robertson,et al.  A comparison of optimal and sub-optimal MAP decoding algorithms operating in the log domain , 1995, Proceedings IEEE International Conference on Communications ICC '95.

[19]  Steffen Barembruch A comparison of approximate Viterbi techniques and particle filtering for data estimation in digital communications , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[20]  G. Kitagawa Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[21]  Rong Chen,et al.  Multilevel Mixture Kalman Filter , 2004, EURASIP J. Adv. Signal Process..

[22]  Luc Vandendorpe,et al.  A Theoretical Framework for Iterative Synchronization Based on the Sum–Product and the Expectation-Maximization Algorithms , 2007, IEEE Transactions on Signal Processing.

[23]  Brian Jefferies Feynman-Kac Formulae , 1996 .

[24]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[25]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[26]  D. Andrews Testing When a Parameter Is on the Boundary of the Maintained Hypothesis , 2001 .

[27]  C. Fragouli,et al.  Channel estimation and equalization in fading , 1999, Conference Record of the Thirty-Third Asilomar Conference on Signals, Systems, and Computers (Cat. No.CH37020).

[28]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[29]  Marc Moonen,et al.  MLSE and MAP Equalization for Transmission Over Doubly Selective Channels , 2009, IEEE Transactions on Vehicular Technology.

[30]  Mónica F. Bugallo,et al.  A sequential Monte Carlo method for adaptive blind timing estimation and data detection , 2005, IEEE Transactions on Signal Processing.

[31]  O. Cappé,et al.  Sequential Monte Carlo smoothing with application to parameter estimation in nonlinear state space models , 2006, math/0609514.

[32]  Jun S. Liu,et al.  Blind Deconvolution via Sequential Imputations , 1995 .

[33]  Laurent Ros,et al.  Joint data QR-detection and Kalman estimation for OFDM time-varying Rayleigh channel complex gains , 2010, IEEE Transactions on Communications.

[34]  John Cocke,et al.  Optimal decoding of linear codes for minimizing symbol error rate (Corresp.) , 1974, IEEE Trans. Inf. Theory.

[35]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[36]  Arnaud Doucet,et al.  Convergence of Sequential Monte Carlo Methods , 2007 .

[37]  Liang Hong,et al.  Maximum likelihood BPSK and QPSK classifier in fading environment using the EM algorithm , 2006, 2006 Proceeding of the Thirty-Eighth Southeastern Symposium on System Theory.

[38]  Jerry M. Mendel,et al.  Identification of nonminimum phase systems using higher order statistics , 1989, IEEE Trans. Acoust. Speech Signal Process..

[39]  N. E. Lay,et al.  Modulation classification of signals in unknown ISI environments , 1995, Proceedings of MILCOM '95.

[40]  Y. Bar-Ness,et al.  Modulation classification in fading channels using antenna arrays , 2004, IEEE MILCOM 2004. Military Communications Conference, 2004..

[41]  Weidong Wang,et al.  Sequential Monte Carlo localization in mobile sensor networks , 2009, Wirel. Networks.

[42]  Franz Hlawatsch,et al.  A compressed sensing technique for OFDM channel estimation in mobile environments: Exploiting channel sparsity for reducing pilots , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[43]  Paolo Giudici,et al.  Likelihood‐Ratio Tests for Hidden Markov Models , 2000, Biometrics.

[44]  Stephen E. Fienberg,et al.  Testing Statistical Hypotheses , 2005 .

[45]  Marcelo G. S. Bruno,et al.  Particle Filters for Joint Blind Equalization and Decoding in Frequency-Selective Channels , 2008, IEEE Transactions on Signal Processing.

[46]  P. Fearnhead,et al.  On‐line inference for hidden Markov models via particle filters , 2003 .

[47]  John B. Anderson,et al.  Sequential Coding Algorithms: A Survey and Cost Analysis , 1984, IEEE Trans. Commun..

[48]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[49]  R. Nowak,et al.  Compressed sensing of wireless channels in time, frequency, and space , 2008, 2008 42nd Asilomar Conference on Signals, Systems and Computers.

[50]  Jeff A. Bilmes,et al.  A gentle tutorial of the em algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models , 1998 .

[51]  C. Richard Johnson,et al.  Bounds for the MSE performance of constant modulus estimators , 2000, IEEE Trans. Inf. Theory.

[52]  A. Doucet,et al.  Maximum a Posteriori Sequence Estimation Using Monte Carlo Particle Filters , 2001, Annals of the Institute of Statistical Mathematics.

[53]  M. Salman Asif,et al.  Random channel coding and blind deconvolution , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[54]  Brian Roark,et al.  Probabilistic Top-Down Parsing and Language Modeling , 2001, CL.

[55]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[56]  D. Godard,et al.  Self-Recovering Equalization and Carrier Tracking in Two-Dimensional Data Communication Systems , 1980, IEEE Trans. Commun..

[57]  Stanley J. Simmons,et al.  Breadth-first trellis decoding with adaptive effort , 1990, IEEE Trans. Commun..

[58]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[59]  Andrew J. Viterbi,et al.  Error bounds for convolutional codes and an asymptotically optimum decoding algorithm , 1967, IEEE Trans. Inf. Theory.

[60]  T. Bertozzi,et al.  On particle filtering for digital communications , 2003, 2003 4th IEEE Workshop on Signal Processing Advances in Wireless Communications - SPAWC 2003 (IEEE Cat. No.03EX689).

[61]  Geert Leus,et al.  Semi-blind channel estimation for rapidly time-varying channels , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[62]  Marc Moonen,et al.  Deterministic subspace based blind channel estimation for doubly-selective channels , 2003, 2003 4th IEEE Workshop on Signal Processing Advances in Wireless Communications - SPAWC 2003 (IEEE Cat. No.03EX689).

[63]  Ghassan Kawas Kaleh,et al.  Joint parameter estimation and symbol detection for linear or nonlinear unknown channels , 1994, IEEE Trans. Commun..

[64]  K. C. Ho,et al.  BPSK and QPSK modulation classification with unknown signal level , 2000, MILCOM 2000 Proceedings. 21st Century Military Communications. Architectures and Technologies for Information Superiority (Cat. No.00CH37155).

[65]  Hoang Nguyen,et al.  The expectation-maximization Viterbi algorithm for blind adaptive channel equalization , 2005, IEEE Transactions on Communications.

[66]  Aurélien Garivier,et al.  On approximate maximum likelihood methods for blind identification: How to copewith the curse of dimensionality , 2008, SPAWC 2008.

[67]  Eric Moulines,et al.  Maximum likelihood blind deconvolution for sparse systems , 2010, 2010 2nd International Workshop on Cognitive Information Processing.

[68]  R. Berangi,et al.  Modulation classification of QAM and PSK from their constellation using Genetic Algorithm and hierarchical clustering , 2008, 2008 3rd International Conference on Information and Communication Technologies: From Theory to Applications.

[69]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[70]  J. Lember,et al.  Adjusted Viterbi training for hidden Markov models , 2007, 0709.2317.

[71]  M. Pitt,et al.  Filtering via Simulation: Auxiliary Particle Filters , 1999 .

[72]  P. Fearnhead,et al.  A sequential smoothing algorithm with linear computational cost. , 2010 .

[73]  Andreas Pitsillides,et al.  Addressing network survivability issues by finding the K-best paths through a trellis graph , 1997, Proceedings of INFOCOM '97.

[74]  P. Bucher,et al.  DNA Binding Specificity of Different STAT Proteins , 2001, The Journal of Biological Chemistry.

[75]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[76]  A. Doucet,et al.  Monte Carlo Smoothing for Nonlinear Time Series , 2004, Journal of the American Statistical Association.

[77]  Haikady N. Nagaraja,et al.  Inference in Hidden Markov Models , 2006, Technometrics.

[78]  Robert D. Nowak,et al.  Compressed Channel Sensing: A New Approach to Estimating Sparse Multipath Channels , 2010, Proceedings of the IEEE.

[79]  A. Paulraj,et al.  Semi-blind channel identification and equalization in OFDM: an expectation-maximization approach , 2002, Proceedings IEEE 56th Vehicular Technology Conference.

[80]  Frédéric Lehmann Blind estimation and detection of space-time trellis coded transmissions over the Rayleigh fading MIMO channel , 2008, IEEE Transactions on Communications.

[81]  A. Doucet,et al.  Smoothing algorithms for state–space models , 2010 .

[82]  A. Scaglione,et al.  Estimation of sparse multipath channels , 2008, MILCOM 2008 - 2008 IEEE Military Communications Conference.

[83]  Jr. G. Forney,et al.  Viterbi Algorithm , 1973, Encyclopedia of Machine Learning.

[84]  Deva K. Borah,et al.  Estimation of time-varying frequency-selective channels using a matching pursuit technique , 2003, 2003 IEEE Wireless Communications and Networking, 2003. WCNC 2003..

[85]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[86]  Philip Schniter,et al.  Low-complexity equalization of OFDM in doubly selective channels , 2004, IEEE Transactions on Signal Processing.

[87]  Philip Schniter,et al.  EM-based soft noncoherent equalization of doubly selective channels using tree search and basis expansion , 2009, 2009 IEEE 10th Workshop on Signal Processing Advances in Wireless Communications.

[88]  R. Douc,et al.  Limit theorems for weighted samples with applications to sequential Monte Carlo methods , 2005, math/0507042.

[89]  G. Kitagawa The two-filter formula for smoothing and an implementation of the Gaussian-sum smoother , 1994 .

[90]  R. Nowak,et al.  Learning sparse doubly-selective channels , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[91]  Bhaskar D. Rao,et al.  Sparse channel estimation via matching pursuit with application to equalization , 2002, IEEE Trans. Commun..

[92]  P.M. Djuric,et al.  Signal processing by particle filtering for binary sensor networks , 2004, 3rd IEEE Signal Processing Education Workshop. 2004 IEEE 11th Digital Signal Processing Workshop, 2004..

[93]  David Q. Mayne,et al.  A solution of the smoothing problem for linear dynamic systems , 1966, Autom..

[94]  Jerry M. Mendel,et al.  Maximum-likelihood classification for digital amplitude-phase modulations , 2000, IEEE Trans. Commun..

[95]  Philip Schniter,et al.  Blind equalization using the constant modulus criterion: a review , 1998, Proc. IEEE.

[96]  Benjamin Friedlander,et al.  Blind equalization of digital communication channels using high-order moments , 1991, IEEE Trans. Signal Process..

[97]  P. Bickel,et al.  Asymptotic normality of the maximum-likelihood estimator for general hidden Markov models , 1998 .

[98]  Jitendra K. Tugnait,et al.  Doubly Selective Channel Estimation Using Exponential Basis Models and Subblock Tracking , 2007, IEEE Transactions on Signal Processing.

[99]  Per Ola Börjesson,et al.  ML estimation of time and frequency offset in OFDM systems , 1997, IEEE Trans. Signal Process..

[100]  Eric Moulines,et al.  The expectation and sparse maximization algorithm , 2010, Journal of Communications and Networks.

[101]  Petar M. Djuric,et al.  Applications of particle filtering to communications: A review , 2002, 2002 11th European Signal Processing Conference.

[102]  Eric Moulines,et al.  Inference in hidden Markov models , 2010, Springer series in statistics.

[103]  Jun S. Liu,et al.  Sequential Imputations and Bayesian Missing Data Problems , 1994 .

[104]  G. David Forney,et al.  Maximum-likelihood sequence estimation of digital sequences in the presence of intersymbol interference , 1972, IEEE Trans. Inf. Theory.

[105]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[106]  Achilleas Anastasopoulos,et al.  Likelihood ratio tests for modulation classification , 2000, MILCOM 2000 Proceedings. 21st Century Military Communications. Architectures and Technologies for Information Superiority (Cat. No.00CH37155).

[107]  Hussein Hijazi Estimation de canal radio-mobile à évolution rapide dans les systèmes à modulation OFMD , 2008 .

[108]  Ali Abdi,et al.  Survey of automatic modulation classification techniques: classical approaches and new trends , 2007, IET Commun..

[109]  Andreas Polydoros,et al.  Likelihood methods for MPSK modulation classification , 1995, IEEE Trans. Commun..

[110]  Branko Ristic,et al.  Beyond the Kalman Filter: Particle Filters for Tracking Applications , 2004 .

[111]  J. A. Catipovic,et al.  Algorithms for joint channel estimation and data recovery-application to equalization in underwater communications , 1991 .

[112]  A. Singer,et al.  Bayesian ML Sequence Detection for ISI Channels , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[113]  P. Moral,et al.  On Adaptive Sequential Monte Carlo Methods , 2008 .

[114]  Carl-Erik W. Sundberg,et al.  List Viterbi decoding algorithms with applications , 1994, IEEE Trans. Commun..

[115]  Jitendra K. Tugnait,et al.  Identification of linear stochastic systems via second- and fourth-order cumulant matching , 1987, IEEE Trans. Inf. Theory.

[116]  Didier Le Ruyet,et al.  Trellis-based search of the maximum a posteriori sequence using particle filtering , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[117]  R. E. Kalman,et al.  New Results in Linear Filtering and Prediction Theory , 1961 .

[118]  Rong Chen,et al.  Adaptive joint detection and decoding in flat-fading channels via mixture Kalman filtering , 2000, IEEE Trans. Inf. Theory.

[119]  Jitendra K. Tugnait,et al.  Detection and estimation for abruptly changing systems , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[120]  George Tzagkarakis,et al.  Bayesian compressed sensing imaging using a Gaussian Scale Mixture , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[121]  Petar M. Djuric,et al.  Blind equalization of frequency-selective channels by sequential importance sampling , 2004, IEEE Transactions on Signal Processing.

[122]  A. Shapiro Asymptotic distribution of test statistics in the analysis of moment structures under inequality constraints , 1985 .

[123]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[124]  Costas N. Georghiades,et al.  Sequence estimation in the presence of random parameters via the EM algorithm , 1997, IEEE Trans. Commun..

[125]  Holger Rauhut,et al.  Compressive Estimation of Doubly Selective Channels in Multicarrier Systems: Leakage Effects and Sparsity-Enhancing Processing , 2009, IEEE Journal of Selected Topics in Signal Processing.

[126]  Hoang Nguyen,et al.  Blind and semi-blind equalization of CPM signals with the EMV algorithm , 2003, IEEE Trans. Signal Process..

[127]  Laurent Ros,et al.  Polynomial Estimation of Time-Varying Multipath Gains With Intercarrier Interference Mitigation in OFDM Systems , 2009, IEEE Transactions on Vehicular Technology.

[128]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[129]  Frederick Jelinek,et al.  Statistical methods for speech recognition , 1997 .

[130]  É. Moulines,et al.  A sparse EM algorithm for blind and semi-blind identification of doubly selective OFDM channels , 2010, 2010 IEEE 11th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[131]  Aurélien Garivier,et al.  On optimal sampling for particle filtering in digital communication , 2008, 2008 IEEE 9th Workshop on Signal Processing Advances in Wireless Communications.

[132]  Jeffrey S. Reeve,et al.  A parallel Viterbi decoder for block cyclic and convolution codes , 2006, Signal Process..

[133]  Jean-Jacques Fuchs Multipath time-delay estimation , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[134]  Dirk T. M. Slock,et al.  Blind fractionally-spaced equalization, perfect-reconstruction filter banks and multichannel linear prediction , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[135]  Octavia A. Dobre,et al.  On the likelihood-based approach to modulation classification , 2009, IEEE Transactions on Wireless Communications.

[136]  M. V. Eyuboglu,et al.  Reduced-state sequence estimation for trellis-coded modulation on intersymbol interference channels , 1988, IEEE Global Telecommunications Conference and Exhibition. Communications for the Information Age.

[137]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[138]  Jun S. Liu,et al.  Sequential Monte Carlo methods for dynamic systems , 1997 .

[139]  David Tse,et al.  Fundamentals of Wireless Communication , 2005 .

[140]  Xiaodong Wang,et al.  Multilevel sequential monte carlo algorithms for MIMO demodulation , 2007, IEEE Transactions on Wireless Communications.

[141]  K. DonaldW. Generalized Method of Moments Estimation When a Parameter Is on a Boundary , 1999 .