Separation of Chaotic Signals by Reservoir Computing

We demonstrate the utility of machine learning in the separation of superimposed chaotic signals using a technique called reservoir computing. We assume no knowledge of the dynamical equations that produce the signals and require only training data consisting of finite-time samples of the component signals. We test our method on signals that are formed as linear combinations of signals from two Lorenz systems with different parameters. Comparing our nonlinear method with the optimal linear solution to the separation problem, the Wiener filter, we find that our method significantly outperforms the Wiener filter in all the scenarios we study. Furthermore, this difference is particularly striking when the component signals have similar frequency spectra. Indeed, our method works well when the component frequency spectra are indistinguishable-a case where a Wiener filter performs essentially no separation.

[1]  Miguel C. Soriano,et al.  Electrocardiogram Classification Using Reservoir Computing With Logistic Regression , 2015, IEEE Journal of Biomedical and Health Informatics.

[2]  Luigi Fortuna,et al.  Separation and synchronization of chaotic signals by optimization. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[4]  Jayanta Basak,et al.  Weather Data Mining Using Independent Component Analysis , 2004, J. Mach. Learn. Res..

[5]  B. Schrauwen,et al.  Reservoir computing and extreme learning machines for non-linear time-series data analysis , 2013, Neural Networks.

[6]  Nikolaos Doukas,et al.  A blind source separation based cryptography scheme for mobile military communication applications , 2008 .

[7]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[8]  Benjamin Schrauwen,et al.  Stable Output Feedback in Reservoir Computing Using Ridge Regression , 2008, ICANN.

[9]  R. Brockett,et al.  Reservoir observers: Model-free inference of unmeasured variables in chaotic systems. , 2017, Chaos.

[10]  Fang Yi,et al.  Blind Separation of Mixing Chaotic Signals Based on ICA Using Kurtosis , 2012, 2012 International Conference on Computer Science and Service System.

[11]  Warwick Tucker,et al.  Foundations of Computational Mathematics a Rigorous Ode Solver and Smale's 14th Problem , 2022 .

[12]  DeLiang Wang,et al.  Supervised Speech Separation Based on Deep Learning: An Overview , 2017, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[13]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[14]  Ken Umeno,et al.  Blind source separation of chaotic laser signals by independent component analysis. , 2008, Optics express.

[15]  Petros Koumoutsakos,et al.  Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks , 2018, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[16]  Roland Badeau,et al.  Singing voice detection with deep recurrent neural networks , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[17]  Teresa Bernarda Ludermir,et al.  Investigating the use of Reservoir Computing for forecasting the hourly wind speed in short -term , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[18]  E. Lorenz.  Deterministic nonperiodic flow , 1963 .

[19]  Jungwon Lee,et al.  Residual LSTM: Design of a Deep Recurrent Architecture for Distant Speech Recognition , 2017, INTERSPEECH.

[20]  José Carlos Príncipe,et al.  Analysis and Design of Echo State Networks , 2007, Neural Computation.

[21]  Jaideep Pathak,et al.  Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach. , 2018, Physical review letters.

[22]  Mikkel N. Schmidt,et al.  Single-channel speech separation using sparse non-negative matrix factorization , 2006, INTERSPEECH.

[23]  Zhi-Hui Hu,et al.  Gradient method for blind chaotic signal separation based on proliferation exponent , 2014 .

[24]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series , 1964 .

[25]  Michelle Girvan,et al.  Similarity Learning and Generalization with Limited Data: A Reservoir Computing Approach , 2018, Complex..

[26]  L. Tsimring,et al.  Multiplexing chaotic signals using synchronization , 1996 .

[27]  DeLiang Wang,et al.  A classification based approach to speech segregation. , 2012, The Journal of the Acoustical Society of America.

[28]  Christian Jutten,et al.  On the blind source separation of human electroencephalogram by approximate joint diagonalization of second order statistics , 2008, Clinical Neurophysiology.

[29]  L. M. Pecora,et al.  Using multiple attractor chaotic systems for communication , 1998, 1998 IEEE International Conference on Electronics, Circuits and Systems. Surfing the Waves of Science and Technology (Cat. No.98EX196).

[30]  Luigi Fortuna,et al.  Separation and synchronization of piecewise linear chaotic systems. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[31]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .