Reviving the Dynamics of Attacked Reservoir Computers

Physically implemented neural networks are subject to external perturbations and internal variations. Existing works focus on the adversarial attacks but seldom consider attack on the network structure and the corresponding recovery method. Inspired by the biological neural compensation mechanism and the neuromodulation technique in clinical practice, we propose a novel framework of reviving attacked reservoir computers, consisting of several strategies direct at different types of attacks on structure by adjusting only a minor fraction of edges in the reservoir. Numerical experiments demonstrate the efficacy and broad applicability of the framework and reveal inspiring insights into the mechanisms. This work provides a vehicle to improve the robustness of reservoir computers and can be generalized to broader types of neural networks.

[1]  Huaqiang Wu,et al.  A memristor-based analogue reservoir computing system for real-time and power-efficient signal processing , 2022, Nature Electronics.

[2]  Huaqiang Wu,et al.  Rotating neurons for all-analog implementation of cyclic reservoir computing , 2022, Nature Communications.

[3]  N. Zhong,et al.  Ultralow‐Power Machine Vision with Self‐Powered Sensor Reservoir , 2022, Advanced science.

[4]  Yuchao Yang,et al.  Multilayer Reservoir Computing Based on Ferroelectric α‐In2Se3 for Hierarchical Information Processing , 2022, Advanced materials.

[5]  Shaojie Qiao,et al.  Small perturbations are enough: Adversarial attacks on time series prediction , 2021, Inf. Sci..

[6]  Ling Shao,et al.  ReCU: Reviving the Dead Weights in Binary Neural Networks , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[7]  Charu Aggarwal,et al.  Adversarial Attacks and Defenses on Graphs , 2021, SIGKDD Explor..

[8]  B. Barzel,et al.  Reviving a failed network through microscopic interventions , 2020, Nature Physics.

[9]  Kazuyuki Aihara,et al.  Autoreservoir computing for multistep ahead prediction based on the spatiotemporal information transformation , 2020, Nature Communications.

[10]  Kazuyuki Aihara,et al.  Common stochastic inputs induce neuronal transient synchronization with partial reset , 2020, Neural Networks.

[11]  Chun Zhang,et al.  Long-term prediction of chaotic systems with machine learning , 2020 .

[12]  F. Koushanfar,et al.  Adversarial Deepfakes: Evaluating Vulnerability of Deepfake Detectors to Adversarial Examples , 2020, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[13]  Fang Wen,et al.  FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping , 2019, ArXiv.

[14]  Jaideep Pathak,et al.  Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics , 2019, Neural Networks.

[15]  Davide Cassi,et al.  Conditional attack strategy for real-world complex networks , 2019, Physica A: Statistical Mechanics and its Applications.

[16]  Allen G Hart,et al.  Embedding and approximation theorems for echo state networks , 2019, Neural Networks.

[17]  Liang Zhao,et al.  Interpreting and Evaluating Neural Network Robustness , 2019, IJCAI.

[18]  Germain Forestier,et al.  Adversarial Attacks on Deep Neural Networks for Time Series Classification , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[19]  I. Sibon,et al.  Transcranial direct current stimulation in post-stroke aphasia rehabilitation: A systematic review. , 2019, Annals of physical and rehabilitation medicine.

[20]  Houshang Darabi,et al.  Adversarial Attacks on Time Series , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Daniel Kroening,et al.  A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretability , 2018, Comput. Sci. Rev..

[22]  Kazuyuki Aihara,et al.  Randomly distributed embedding making short-term high-dimensional data predictable , 2018, Proceedings of the National Academy of Sciences.

[23]  Stephan Günnemann,et al.  Adversarial Attacks on Neural Networks for Graph Data , 2018, KDD.

[24]  Zhi Yan,et al.  Quantifying Importance of Edges in Networks , 2018, IEEE Transactions on Circuits and Systems II: Express Briefs.

[25]  Kouichi Sakurai,et al.  One Pixel Attack for Fooling Deep Neural Networks , 2017, IEEE Transactions on Evolutionary Computation.

[26]  Tal Hassner,et al.  On Face Segmentation, Face Swapping, and Face Perception , 2017, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[27]  Silvia Conforto,et al.  Assessing cortical synchronization during transcranial direct current stimulation: A graph-theoretical analysis , 2016, NeuroImage.

[28]  Song Han,et al.  Learning both Weights and Connections for Efficient Neural Network , 2015, NIPS.

[29]  F. Cicchetti,et al.  Cellular and Molecular Mechanisms of Action of Transcranial Direct Current Stimulation: Evidence from In Vitro and In Vivo Models , 2015, International Journal of Neuropsychopharmacology.

[30]  Mary E. Meyerand,et al.  Age-Related Reorganizational Changes in Modularity and Functional Connectivity of Human Brain Networks , 2014, Brain Connect..

[31]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[32]  Michele Bellingeri,et al.  Efficiency of attack strategies on complex model and real-world networks , 2013, ArXiv.

[33]  Stefan J. Kiebel,et al.  Re-visiting the echo state property , 2012, Neural Networks.

[34]  L. Merabet,et al.  Clinical research with transcranial direct current stimulation (tDCS): Challenges and future directions , 2012, Brain Stimulation.

[35]  Fei Xue,et al.  Structural vulnerability of power systems: A topological approach , 2011 .

[36]  M. Bikson,et al.  Electrode Positioning and Montage in Transcranial Direct Current Stimulation , 2011, Journal of visualized experiments : JoVE.

[37]  Benjamin Schrauwen,et al.  Memory versus non-linearity in reservoirs , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[38]  John Skvoretz,et al.  Node centrality in weighted networks: Generalizing degree and shortest paths , 2010, Soc. Networks.

[39]  Á. Pascual-Leone,et al.  Treatment of depression with transcranial direct current stimulation (tDCS): A Review , 2009, Experimental Neurology.

[40]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[41]  W. Maass,et al.  State-dependent computations: spatiotemporal processing in cortical networks , 2009, Nature Reviews Neuroscience.

[42]  Herbert Jaeger,et al.  Optimization and applications of echo state networks with leaky- integrator neurons , 2007, Neural Networks.

[43]  Benjamin Schrauwen,et al.  Reservoir-based techniques for speech recognition , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[44]  E. Marder,et al.  Variability, compensation and homeostasis in neuron and network function , 2006, Nature Reviews Neuroscience.

[45]  J. A. Rodríguez-Velázquez,et al.  Subgraph centrality in complex networks. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[46]  Wenpu Xing,et al.  Weighted PageRank algorithm , 2004, Proceedings. Second Annual Conference on Communication Networks and Services Research, 2004..

[47]  M. Barthelemy.  Betweenness centrality in large complex networks , 2003, cond-mat/0309436.

[48]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[49]  Xiao-Dong Zhang,et al.  On the Spectral Radius of Graphs with Cut Vertices , 2001, J. Comb. Theory, Ser. B.

[50]  S. Havlin,et al.  Breakdown of the internet under intentional attack. , 2000, Physical review letters.

[51]  A. Barabasi,et al.  Error and attack tolerance of complex networks , 2000, Nature.

[52]  Sorin Draghici,et al.  Neural Networks in Analog Hardware - Design and Implementation Issues , 2000, Int. J. Neural Syst..

[53]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[54]  Chidchanok Lursinsap,et al.  Weight shifting techniques for self-recovery neural networks , 1994, IEEE Trans. Neural Networks.

[55]  D. Meyer,et al.  A Neural System for Error Detection and Compensation , 1993 .

[56]  J. Bolland,et al.  Sorting out centrality: An analysis of the performance of four centrality models in real and simulated networks , 1988 .

[57]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[58]  E. Lorenz.  Deterministic nonperiodic flow , 1963 .

[59]  Huanfei Ma,et al.  Global optimization of hyper-parameters in reservoir computing , 2022, Electronic Research Archive.

[60]  H. Jaeger,et al.  Unconventional Information Processing Systems , Novel Hardware : A Tour d ’ Horizon , 2017 .

[61]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[62]  Peter Rossmanith,et al.  Local search algorithms , 2008 .

[63]  Henry Markram,et al.  A Model for Real-Time Computation in Generic Neural Microcircuits , 2002, NIPS.