Link inference of noisy delay-coupled networks: Machine learning and opto-electronic experimental tests

We devise a machine learning technique to solve the general problem of inferring network links that have time-delays. The goal is to do this purely from time-series data of the network nodal states. This task has applications in fields ranging from applied physics and engineering to neuroscience and biology. To achieve this, we first train a type of machine learning system known as reservoir computing to mimic the dynamics of the unknown network. We formulate and test a technique that uses the trained parameters of the reservoir system output layer to deduce an estimate of the unknown network structure. Our technique, by its nature, is non-invasive, but is motivated by the widely-used invasive network inference method whereby the responses to active perturbations applied to the network are observed and employed to infer network links (e.g., knocking down genes to infer gene regulatory networks). We test this technique on experimental and simulated data from delay-coupled opto-electronic oscillator networks. We show that the technique often yields very good results particularly if the system does not exhibit synchrony. We also find that the presence of dynamical noise can strikingly enhance the accuracy and ability of our technique, especially in networks that exhibit synchrony.

[1]  Edward Ott,et al.  Attractor reconstruction by machine learning. , 2018, Chaos.

[2]  S Yanchuk,et al.  Synchronizing distant nodes: a universal classification of networks. , 2010, Physical review letters.

[3]  Stefano Allesina,et al.  Ecological Network Inference From Long-Term Presence-Absence Data , 2017, Scientific Reports.

[4]  Colin M Beale,et al.  Revealing ecological networks using Bayesian network inference algorithms. , 2010, Ecology.

[5]  John Quackenbush,et al.  Inference and validation of predictive gene networks from biomedical literature and gene expression data. , 2014, Genomics.

[6]  Erik Bollt,et al.  On explaining the surprising success of reservoir computing forecaster of chaos? The universal machine learning dynamical system with contrast to VAR and DMD. , 2020, Chaos.

[7]  Michelle Girvan,et al.  Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model , 2018, Chaos.

[8]  Laurent Larger,et al.  Chaos-based communications at high bit rates using commercial fibre-optic links , 2005, Nature.

[9]  Jason Z. Kim,et al.  Supervised chaotic source separation by a tank of water , 2020, Chaos.

[10]  Wen-Xu Wang,et al.  Noise bridges dynamical correlation and topology in coupled oscillator networks. , 2010, Physical review letters.

[11]  Edward Ott,et al.  Complex dynamics and synchronization of delayed-feedback nonlinear oscillators , 2009, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[12]  Tiago P. Peixoto Network Reconstruction and Community Detection from Dynamics , 2019, Physical review letters.

[13]  Serge Massar,et al.  Fully analogue photonic reservoir computer , 2016, Scientific Reports.

[14]  Schreiber,et al.  Measuring information transfer , 2000, Physical review letters.

[15]  Ralph G Andrzejak,et al.  Inferring directed networks using a rank-based connectivity measure. , 2019, Physical review. E.

[16]  Jaideep Pathak,et al.  Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach. , 2018, Physical review letters.

[17]  Zoran Levnajic,et al.  Reconstructing dynamical networks via feature ranking , 2019, Chaos.

[18]  Dane Taylor,et al.  Causal Network Inference by Optimal Causation Entropy , 2014, SIAM J. Appl. Dyn. Syst..

[19]  A. Barabasi,et al.  Interactome Networks and Human Disease , 2011, Cell.

[20]  J Martinerie,et al.  Functional modularity of background activities in normal and epileptic brain networks. , 2008, Physical review letters.

[21]  L. Tsimring,et al.  Generalized synchronization of chaos in directionally coupled chaotic systems. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[22]  Adilson E Motter,et al.  Heterogeneity in oscillator networks: are smaller worlds easier to synchronize? , 2003, Physical review letters.

[23]  Serge Massar,et al.  High performance photonic reservoir computer based on a coherently driven passive cavity , 2015, ArXiv.

[24]  Alan V. Oppenheim,et al.  Discrete-time Signal Processing. Vol.2 , 2001 .

[25]  Yanne K Chembo,et al.  Machine learning based on reservoir computing with time-delayed optoelectronic and photonic systems. , 2020, Chaos.

[26]  David Cai,et al.  Causal and structural connectivity of pulse-coupled nonlinear networks. , 2013, Physical review letters.

[27]  Lucas Illing,et al.  Isochronal chaos synchronization of delay-coupled optoelectronic oscillators. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[28]  Eckehard Schöll,et al.  Experimental observations of group synchrony in a system of chaotic optoelectronic oscillators. , 2013, Physical review letters.

[29]  Siyang Leng,et al.  Reconstructing directional causal networks with random forest: Causality meeting machine learning. , 2019, Chaos.

[30]  Jianping Yao,et al.  Optoelectronic Oscillators for High Speed and High Resolution Optical Sensing , 2017, Journal of Lightwave Technology.

[31]  Moritz Helias,et al.  Correlated fluctuations in strongly-coupled binary networks beyond equilibrium , 2015, 1512.01073.

[32]  Jürgen Kurths,et al.  Identifying causal gateways and mediators in complex spatio-temporal systems , 2015, Nature Communications.

[33]  L Pesquera,et al.  Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. , 2012, Optics express.

[34]  Benjamin Schrauwen,et al.  Optoelectronic Reservoir Computing , 2011, Scientific Reports.

[35]  Y. Takane,et al.  Generalized Inverse Matrices , 2011 .

[36]  J Kurths,et al.  Transition from phase to generalized synchronization in time-delay systems. , 2008, Chaos.

[37]  R. Albert Network Inference, Analysis, and Modeling in Systems Biology , 2007, The Plant Cell Online.

[38]  Adam P. Rosebrock,et al.  A global genetic interaction network maps a wiring diagram of cellular function , 2016, Science.

[39]  Percy Venegas Tracing the untraceable: AI network inference for the dark web and crypto privacy coins , 2018 .

[40]  Michelle Girvan,et al.  Separation of Chaotic Signals by Reservoir Computing , 2020, Chaos.

[41]  Jianfeng Feng,et al.  Granger causality vs. dynamic Bayesian network inference: a comparative study , 2009, BMC Bioinformatics.

[42]  Benjamin Schrauwen,et al.  Compact hardware liquid state machines on FPGA for real-time speech recognition , 2008, Neural Networks.

[43]  Bin Xu,et al.  Model reconstruction from temporal data for coupled oscillator networks. , 2019, Chaos.

[44]  Damien Querlioz,et al.  Physics for neuromorphic computing , 2020, Nature Reviews Physics.

[45]  Adilson E Motter,et al.  Robustness of optimal synchronization in real networks. , 2011, Physical review letters.

[46]  Guillermo A. Cecchi,et al.  Noise-Driven Causal Inference in Biomolecular Networks , 2015, PloS one.

[47]  Ulrich Parlitz,et al.  Observing spatio-temporal dynamics of excitable media using reservoir computing. , 2018, Chaos.

[48]  Abhranil Das,et al.  Systematic errors in connectivity inferred from activity in strongly recurrent networks , 2020, Nature neuroscience.

[49]  Xiao Han,et al.  Robust Reconstruction of Complex Networks from Sparse Data , 2015, Physical review letters.

[50]  Piet Van Mieghem,et al.  Epidemic processes in complex networks , 2014, ArXiv.

[51]  C. Myers,et al.  Genetic interaction networks: toward an understanding of heritability. , 2013, Annual review of genomics and human genetics.

[52]  Naftali Tishby,et al.  Machine learning and the physical sciences , 2019, Reviews of Modern Physics.

[53]  O. Sporns,et al.  Network neuroscience , 2017, Nature Neuroscience.

[54]  S. Strogatz,et al.  Dense networks that do not synchronize and sparse ones that do. , 2019, Chaos.

[55]  Sean N. Brennan,et al.  Observability and Controllability of Nonlinear Networks: The Role of Symmetry , 2013, Physical review. X.

[56]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[57]  Marios Mattheakis,et al.  Machine Learning With Observers Predicts Complex Spatiotemporal Behavior , 2018, Front. Phys..

[58]  Vladimir A. Maksimenko,et al.  Feed-forward artificial neural network provides data-driven inference of functional connectivity. , 2019, Chaos.

[59]  T. E. Hull,et al.  Random Number Generators , 1962 .

[60]  Laurent Larger,et al.  Chaotic breathers in delayed electro-optical systems. , 2005, Physical review letters.

[61]  Xiao-Ke Xu,et al.  Network embedding for link prediction: The pitfall and improvement. , 2019, Chaos.

[62]  T. Sauer,et al.  Reconstructing the topology of sparsely connected dynamical networks. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[63]  Laurent Larger,et al.  High-Speed Photonic Reservoir Computing Using a Time-Delay-Based Architecture: Million Words per Second Classification , 2017 .

[64]  Jaideep Pathak,et al.  Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data. , 2017, Chaos.

[65]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[66]  Laurent Larger,et al.  Optoelectronic oscillators with time-delayed feedback , 2019, Reviews of Modern Physics.

[67]  Junichiro Yoshimoto,et al.  Connectivity inference from neural recording data: Challenges, mathematical bases and research directions , 2017, Neural Networks.

[68]  Wen-Xu Wang,et al.  Reverse engineering of complex dynamical networks in the presence of time-delayed interactions based on noisy time series. , 2011, Chaos.

[69]  Joseph D. Hart,et al.  Experimental observation of chimera and cluster states in a minimal globally coupled network. , 2015, Chaos.

[70]  Joseph D. Hart,et al.  Adding connections can hinder network synchronization of time-delayed oscillators. , 2015, Physical review. E, Statistical, nonlinear, and soft matter physics.

[71]  L. Maleki,et al.  Optoelectronic microwave oscillator , 1996 .

[72]  N. Price,et al.  Biochemical and statistical network models for systems biology. , 2007, Current opinion in biotechnology.

[73]  S. Bressler,et al.  Granger Causality: Basic Theory and Application to Neuroscience , 2006, q-bio/0608035.

[74]  Weiqing Wang,et al.  Perturbation Biology: Inferring Signaling Networks in Cellular Systems , 2013, PLoS Comput. Biol..

[75]  Toshiyuki Yamane,et al.  Recent Advances in Physical Reservoir Computing: A Review , 2018, Neural Networks.

[76]  Francesco Sorrentino,et al.  Cluster synchronization and isolated desynchronization in complex networks with symmetries , 2013, Nature Communications.

[77]  Michael Small,et al.  The reservoir's perspective on generalized synchronization. , 2019, Chaos.

[78]  Neo D. Martinez,et al.  Food-web structure and network theory: The role of connectance and size , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[79]  Tiago P. Peixoto Reconstructing networks with unknown and heterogeneous errors , 2018, Physical Review X.

[80]  Hana El-Samad,et al.  Using Dynamic Noise Propagation to Infer Causal Regulatory Relationships in Biochemical Networks , 2014, ACS synthetic biology.

[81]  M. Timme,et al.  Inferring Network Connectivity from Event Timing Patterns. , 2018, Physical review letters.

[82]  Eckehard Schöll,et al.  Broadband chaos generated by an optoelectronic oscillator. , 2009, Physical review letters.

[83]  A. W. M. van den Enden,et al.  Discrete Time Signal Processing , 1989 .

[84]  Parlitz,et al.  Generalized synchronization, predictability, and equivalence of unidirectionally coupled dynamical systems. , 1996, Physical review letters.

[85]  Joseph D. Hart,et al.  Experiments on networks of coupled opto-electronic oscillators and physical random number generators , 2018 .

[86]  Laurent Larger,et al.  Delayed dynamical systems: networks, chimeras and reservoir computing , 2018, Philosophical Transactions of the Royal Society A.

[87]  Xiang Lin,et al.  Generative dynamic link prediction. , 2019, Chaos.

[88]  L. Appeltant,et al.  Information processing using a single dynamical node as complex system , 2011, Nature communications.

[89]  E. Bullmore,et al.  Behavioral / Systems / Cognitive Functional Connectivity and Brain Networks in Schizophrenia , 2010 .

[90]  Marc Timme,et al.  Breaking synchrony by heterogeneity in complex networks. , 2003, Physical review letters.

[91]  Chao Sima,et al.  Inference of Gene Regulatory Networks Using Time-Series Data: A Survey , 2009, Current genomics.

[92]  Amitava Banerjee,et al.  Using Machine Learning to Assess Short Term Causal Dependence and Infer Network Links , 2019, Chaos.

[93]  J. Kurths,et al.  Global generalized synchronization in networks of different time-delay systems , 2013, 1301.7590.

[94]  Laurent Larger,et al.  Complexity in electro-optic delay dynamics: modelling, design and applications , 2013, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.