Adaptive filtering with the self-organizing map: A performance comparison

In this paper we provide an in-depth evaluation of the SOM as a feasible tool for nonlinear adaptive filtering. A comprehensive survey of existing SOM-based and related architectures for learning input-output mappings is carried out and the application of these architectures to nonlinear adaptive filtering is formulated. Then, we introduce two simple procedures for building RBF-based nonlinear filters using the Vector-Quantized Temporal Associative Memory (VQTAM), a recently proposed method for learning dynamical input-output mappings using the SOM. The aforementioned SOM-based adaptive filters are compared with standard FIR/LMS and FIR/LMS-Newton linear transversal filters, as well as with powerful MLP-based filters in nonlinear channel equalization and inverse modeling tasks. The obtained results in both tasks indicate that SOM-based filters can consistently outperform powerful MLP-based ones.

[1]  Chi K. Tse,et al.  A neural-network-based channel-equalization strategy for chaos-based communication systems , 2003 .

[2]  Yu-Geng Xi,et al.  Nonlinear system modeling by competitive learning and adaptive fuzzy inference system , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[3]  M. V. Velzen,et al.  Self-organizing maps , 2007 .

[4]  Helge J. Ritter,et al.  Self-Organizing Feature Maps for Modeling and Control of Robotic Manipulators , 2003, J. Intell. Robotic Syst..

[5]  Helge J. Ritter,et al.  Rapid learning with parametrized self-organizing maps , 1996, Neurocomputing.

[6]  Wolfgang RosenstielLehrstuhl.  Topology-preserving Interpolation in Self-organizing Maps , 1993 .

[7]  Adel A. M. Saleh,et al.  Frequency-Independent and Frequency-Dependent Nonlinear Models of TWT Amplifiers , 1981, IEEE Trans. Commun..

[8]  Thomas Villmann,et al.  Applications of the growing self-organizing map , 1998, Neurocomputing.

[9]  Anthony Zaknich,et al.  A practical sub-space adaptive filter , 2003, Neural Networks.

[10]  Anthony Zaknich,et al.  Introduction to the modified probabilistic neural network for general signal processing applications , 1998, IEEE Trans. Signal Process..

[11]  S. Thomas Alexander,et al.  Adaptive Signal Processing , 1986, Texts and Monographs in Computer Science.

[12]  Allen Gersho,et al.  Competitive learning and soft competition for vector quantizer design , 1992, IEEE Trans. Signal Process..

[13]  Akira Hirose,et al.  Predictive self-organizing map for vector quantization of migratory signals and its application to mobile communications , 2003, IEEE Trans. Neural Networks.

[14]  Mohamed Ibnkahla,et al.  Statistical analysis of a two-layer backpropagation algorithm used for modeling nonlinear memoryless channels: the single neuron case , 1997, IEEE Trans. Signal Process..

[15]  Klaus Schulten,et al.  Topology-conserving maps for learning visuo-motor-coordination , 1989, Neural Networks.

[16]  Simon Haykin,et al.  Adaptive filter theory (2nd ed.) , 1991 .

[17]  V. Kvasnicka,et al.  Neural and Adaptive Systems: Fundamentals Through Simulations , 2001, IEEE Trans. Neural Networks.

[18]  Elias S. Manolakos,et al.  Using recurrent neural networks for adaptive communication channel equalization , 1994, IEEE Trans. Neural Networks.

[19]  I C G Campbell,et al.  European Symposium on Artificial Neural Networks ESANN '95 , 1995 .

[20]  M. Ibnkahla,et al.  Self-organizing maps for rapidly fading nonlinear channel equalization , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[21]  Stephen Grossberg,et al.  ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network , 1991, [1991 Proceedings] IEEE Conference on Neural Networks for Ocean Engineering.

[22]  Olli Simula,et al.  Neural detection of QAM signal with strongly nonlinear receiver , 1998, Neurocomputing.

[23]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[24]  Michel Verleysen,et al.  Double quantization of the regressor space for long-term time series prediction: method and proof of stability , 2004, Neural Networks.

[25]  Xiaoqui Wang.  Detection of Nonlinearly Distorted M-ary QAM Signals Using Self-Organizing Map , 2001 .

[26]  E. Chng,et al.  Reduced complexity implementation of Bayesian equaliser using local RBF network for channel equalisation problem , 1996 .

[27]  J. Príncipe,et al.  Local dynamic modeling with self-organizing maps and applications to nonlinear system identification and control , 1998, Proc. IEEE.

[28]  Xiao Liu,et al.  Conditional distribution learning with neural networks and its application to channel equalization , 1997, IEEE Trans. Signal Process..

[29]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[30]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[31]  T. Kohonen,et al.  Bibliography of Self-Organizing Map SOM) Papers: 1998-2001 Addendum , 2003 .

[32]  Wolfgang Rosenstiel,et al.  Topological interpolation in SOM by affine transformations , 1995, ESANN.

[33]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[34]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[35]  J.E. Mazo,et al.  Digital communications , 1985, Proceedings of the IEEE.

[36]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[37]  Mohamed Ibnkahla,et al.  Applications of neural networks to digital communications - a survey , 2000, Signal Process..

[38]  Donald F. Specht,et al.  The general regression neural network - Rediscovered , 1993, Neural Networks.

[39]  Aluizio F. R. Araújo,et al.  Identification and control of dynamical systems using the self-organizing map , 2004, IEEE Transactions on Neural Networks.

[40]  Bhaskar D. Rao,et al.  Fast adaptive digital equalization by recurrent neural networks , 1997, IEEE Trans. Signal Process..

[41]  Jr. C.Richard Johnson,et al.  On the interaction of adaptive filtering, identification, and control , 1995, IEEE Signal Process. Mag..

[42]  João Cesar M. Mota,et al.  Nonstationary Time Series Prediction Using Local Models Based on Competitive Neural Networks , 2004, IEA/AIE.

[43]  Ganapati Panda,et al.  Nonlinear channel equalization for QAM signal constellation using artificial neural networks , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[44]  Robert Hecht-Nielsen,et al.  Applications of counterpropagation networks , 1988, Neural Networks.

[45]  Michel Verleysen,et al.  Forecasting electricity consumption using nonlinear projection and self-organizing maps , 2002, Neurocomputing.

[46]  D. T. Pham,et al.  Supervised adaptive resonance theory neural network for modelling dynamic systems , 1995, 1995 IEEE International Conference on Systems, Man and Cybernetics. Intelligent Systems for the 21st Century.

[47]  Jose C. Principe,et al.  Neural and Adaptive Systems: Fundamentals through Simulations with CD-ROM , 1999 .

[48]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[49]  J. G. Proakis,et al.  Adaptive equalization with neural networks: new multi-layer perceptron structures and their evaluation , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[50]  Lennart Ljung,et al.  Nonlinear black-box modeling in system identification: a unified overview , 1995, Autom..

[51]  Amaury Lendasse,et al.  Time series forecasting with SOM and local non-linear models - Application to the DAX30 index prediction , 2003 .

[52]  Bernard Widrow,et al.  30 years of adaptive neural networks: perceptron, Madaline, and backpropagation , 1990, Proc. IEEE.

[53]  S. Qureshi,et al.  Adaptive equalization , 1982, Proceedings of the IEEE.

[54]  Jukka Heikkonen,et al.  Time Series Predicition using Recurrent SOM with Local Linear Models , 1997 .

[55]  Michel Verleysen,et al.  On the Use of Self-Organizing Maps to Accelerate Vector Quantization , 2002, Neurocomputing.

[56]  Daniel Roviras,et al.  Equalization of satellite mobile communication channels using combined self-organizing maps and RBF networks , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[57]  J. Sjöberg.  Neural networks for modelling and control of dynamic systems: M. Nørgaard, O. Ravn, N. K. Poulsen and L. K. Hansen. Springer-Verlag, London Berlin Heidelberg, 2000, pp. xiv+246 , 2004 .

[58]  Azzedine Zerguine,et al.  Multilayer perceptron-based DFE with lattice structure , 2001, IEEE Trans. Neural Networks.

[59]  B. Yegnanarayana,et al.  Artificial Neural Networks , 2004 .

[60]  Arthur Flexer,et al.  On the use of self-organizing maps for clustering and visualization , 1999, Intell. Data Anal..

[61]  Bernard Widrow,et al.  Statistical efficiency of adaptive algorithms , 2003, Neural Networks.

[62]  Sheng Chen,et al.  A clustering technique for digital communications channel equalization using radial basis function networks , 1993, IEEE Trans. Neural Networks.

[63]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[64]  Niels Kjølstad Poulsen,et al.  Neural Networks for Modelling and Control of Dynamic Systems: A Practitioner’s Handbook , 2000 .

[65]  Joachim M. Buhmann,et al.  Competitive learning algorithms for robust vector quantization , 1998, IEEE Trans. Signal Process..

[66]  Erkki Oja,et al.  Engineering applications of the self-organizing map , 1996, Proc. IEEE.

[67]  Narasimhan Sundararajan,et al.  Communication channel equalization using complex-valued minimal radial basis function neural networks , 2002, IEEE Trans. Neural Networks.

[68]  Po-Rong Chang,et al.  Adaptive Decision Feedback Equalization for Digital Satellite Channels Using Multilayer Neural Networks , 1995, IEEE J. Sel. Areas Commun..

[69]  John A. Hertz,et al.  Exploiting Neurons with Localized Receptive Fields to Learn Chaos , 1990, Complex Syst..

[70]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[71]  Takeshi Yamakawa,et al.  Self-Organizing Relationship (SOR) Network , 1999 .

[72]  Jörg A. Walter,et al.  Nonlinear prediction with self-organizing maps , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[73]  Howard C. Card,et al.  Competitive Learning Algorithms and Neurocomputer Architecture , 1998, IEEE Trans. Computers.

[74]  Chrysostomos L. Nikias,et al.  Adaptive equalization for PAM and QAM signals with neural networks , 1991, [1991] Conference Record of the Twenty-Fifth Asilomar Conference on Signals, Systems & Computers.