Blind Source Recovery: A Framework in the State Space

Blind Source Recovery (BSR) denotes recovery of original sources/signals from environments that may include convolution, temporal variation, and even nonlinearity. It also infers the recovery of sources even in the absence of precise environment identifiability. This paper describes, in a comprehensive fashion, a generalized BSR formulation achieved by the application of stochastic optimization principles to the Kullback-Liebler divergence as a performance functional subject to the constraints of the general (i.e., nonlinear and time-varying) state space representation. This technique is used to derive update laws for nonlinear time-varying dynamical systems, which are subsequently specialized to time-invariant and linear systems. Further, the state space demixing network structures have been exploited to develop learning rules, capable of handling most filtering paradigms, which can be conveniently extended to nonlinear models. In the special cases, distinct linear state-space algorithms are presented for the minimum phase and non-minimum phase mixing environment models. Conventional (FIR/IIR) filtering models are subsequently derived from this general structure and are compared with material in the recent literature. Illustrative simulation examples are presented to demonstrate the online adaptation capabilities of the developed algorithms. Some of this reported work has also been implemented in dedicated hardware/software platforms.

[1]  Khurram Waheed,et al.  State space blind source recovery for mixtures of multiple source distributions , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[2]  Khurram Waheed,et al.  State Space Feedforward and Feedback Structures for Blind Source Recovery , 2001 .

[3]  Liqing Zhang,et al.  Semiparametric Approach to Multichannel Blind Deconvolution of Nonminimum Phase Systems , 1999, NIPS.

[4]  Reinhold Orglmeister,et al.  Blind source separation of real world signals , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[5]  Andrzej Cichocki,et al.  Multichannel blind deconvolution of non-minimum phase systems using information backpropagation , 1999, ICONIP'99. ANZIIS'99 & ANNES'99 & ACNN'99. 6th International Conference on Neural Information Processing. Proceedings (Cat. No.99EX378).

[6]  Russell H. Lambert,et al.  Blind separation of multiple speakers in a multipath environment , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[7]  Shun-ichi Amari,et al.  Blind source separation-semiparametric statistical approach , 1997, IEEE Trans. Signal Process..

[8]  Khurram Waheed,et al.  Blind source recovery: some implementation and performance issues , 2001, Proceedings of the 44th IEEE 2001 Midwest Symposium on Circuits and Systems. MWSCAS 2001 (Cat. No.01CH37257).

[9]  H. H. Yang,et al.  Information backpropagation for blind separation of sources in nonlinear mixture , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[10]  Shun-ichi Amari,et al.  Neural Learning in Structured Parameter Spaces - Natural Riemannian Gradient , 1996, NIPS.

[11]  Andrzej Cichocki,et al.  Relationships between instantaneous blind source separation and multichannel blind deconvolution , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[12]  Khurram Waheed,et al.  State space blind source recovery of non-minimum phase environments , 2002, The 2002 45th Midwest Symposium on Circuits and Systems, 2002. MWSCAS-2002..

[13]  順一 井村,et al.  Nonlinear Systems Third Edition, Hassan K. Khalil著, 出版社 Prentice Hall, 発行 2002年, 全ページ 750頁, 価格 ￡39.99, ISBN 0-13-067389-7 , 2003 .

[14]  Kari Torkkola,et al.  Blind separation of convolved sources based on information maximization , 1996, Neural Networks for Signal Processing VI. Proceedings of the 1996 IEEE Signal Processing Society Workshop.

[15]  A. Benveniste,et al.  Robust identification of a nonminimum phase system: Blind adjustment of a linear equalizer in data communications , 1980 .

[16]  Fathi M. A. Salam,et al.  Blind source recovery using an adaptive generalized Gaussian score function , 2002, The 2002 45th Midwest Symposium on Circuits and Systems, 2002. MWSCAS-2002..

[17]  A. J. Bell,et al.  A Unifying Information-Theoretic Framework for Independent Component Analysis , 2000 .

[18]  R. Fisher.  The Advanced Theory of Statistics , 1943, Nature.

[19]  Fathi M. A. Salam,et al.  Blind source recovery: algorithms for static and dynamic environments , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[20]  Gamze Erten,et al.  Blind Signal Separation and Recovery in Dynamic Environments , 1997 .

[21]  Fathi M. A. Salam,et al.  Separation of mixed signals in dynamic environments: formulation and some implementation , 1994, Proceedings of 1994 37th Midwest Symposium on Circuits and Systems.

[22]  Fathi M. A. Salam,et al.  Voice extraction by on-line signal separation and recovery , 1999 .

[23]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[24]  Te-Won Lee,et al.  Blind Separation of Delayed and Convolved Sources , 1996, NIPS.

[25]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[26]  Fathi M. A. Salam,et al.  Implementation and test results of a chip for the separation of mixed signals , 1995, Proceedings of ISCAS'95 - International Symposium on Circuits and Systems.

[27]  Fathi M. Salem,et al.  Blind Multi User Detection in DS-CDMA Systems using Natural Gradient based Symbol Recovery Structures , 2003 .

[28]  Fathi M. A. Salam,et al.  Exact entropy series representation for blind source separation , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[29]  S.C. Douglas,et al.  On the relationship between blind deconvolution and blind source separation , 1997, Conference Record of the Thirty-First Asilomar Conference on Signals, Systems and Computers (Cat. No.97CB36136).

[30]  Shun-ichi Amari,et al.  Neural Network Models for Blind Separation of Time Delayed and Convolved Signals , 1997 .

[31]  Fathi M. A. Salam,et al.  An adaptive network for blind separation of independent signals , 1993, 1993 IEEE International Symposium on Circuits and Systems.

[32]  R. Lambert.  Multichannel blind deconvolution: FIR matrix algebra and separation of multipath mixtures , 1996 .

[33]  Fathi M. A. Salam,et al.  The state space framework for blind dynamic signal extraction and recovery , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[34]  Fathi M. A. Salam,et al.  A data-derived quadratic independence measure for adaptive blind source recovery in practical applications , 2002, The 2002 45th Midwest Symposium on Circuits and Systems, 2002. MWSCAS-2002..

[35]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .

[36]  Andrzej Cichocki,et al.  Stability Analysis of Learning Algorithms for Blind Source Separation , 1997, Neural Networks.

[37]  Khurram Waheed,et al.  Cascaded structures for blind source recovery , 2002, The 2002 45th Midwest Symposium on Circuits and Systems, 2002. MWSCAS-2002..

[38]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[39]  A. Cichocki,et al.  Blind Deconvolution of Dynamical Systems:A State-Space Approach , 2000 .

[40]  Christian Jutten,et al.  Space or time adaptive signal processing by neural network models , 1987 .

[41]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .

[42]  Fathi M. A. Salam,et al.  Voice output extraction by signal separation , 1998, ISCAS '98. Proceedings of the 1998 IEEE International Symposium on Circuits and Systems (Cat. No.98CH36187).

[43]  Andrzej Cichocki,et al.  Adaptive blind signal and image processing , 2002 .

[44]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[45]  S.C. Douglas,et al.  Multichannel blind deconvolution and equalization using the natural gradient , 1997, First IEEE Signal Processing Workshop on Signal Processing Advances in Wireless Communications.

[46]  H. Yangy,et al.  Information Back-propagation for Blind Separation of Sources in Non-linear Mixture , 1997 .

[47]  Fathi M. Salem,et al.  New hyperbolic source density models for blind source recovery score functions , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[48]  G. Erten,et al.  Formulation and algorithms for blind signal recovery , 1997, Proceedings of 40th Midwest Symposium on Circuits and Systems. Dedicated to the Memory of Professor Mac Van Valkenburg.

[49]  Fathi M. A. Salam,et al.  Algorithms for blind signal separation and recovery in static and dynamic environments , 1997, Proceedings of 1997 IEEE International Symposium on Circuits and Systems. Circuits and Systems in the Information Age ISCAS '97.

[50]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[51]  K. T. Compton,et al.  American Institute of Physics , 1931, Nature.

[52]  Fathi M. A. Salam,et al.  Real-time separation of audio signals using digital signal processors , 1999 .

[53]  Shun-ichi Amari,et al.  Stability Analysis Of Adaptive Blind Source Separation , 1997 .

[54]  Shun-ichi Amari,et al.  Differential-geometrical methods in statistics , 1985 .

[55]  Juha Karhunen,et al.  Neural approaches to independent component analysis and source separation , 1996, ESANN.