Reservoir computing approaches to recurrent neural network training

[1]  Manvendra Singh,et al.  Speech Recognition Using Neural Networks , 2011 .

[2]  Helmut Hauser,et al.  Echo state networks with filter neurons and a delay&sum readout , 2010, Neural Networks.

[3]  R. Miikkulainen.  Hopfield Network , 2010, Encyclopedia of Machine Learning and Data Mining.

[4]  Robert A. Legenstein,et al.  Spiking Neurons Can Learn to Solve Information Bottleneck Problems and Extract Independent Components , 2009, Neural Computation.

[5]  Luís A. Alexandre,et al.  Reservoir computing for static pattern recognition , 2009, ESANN.

[6]  Minoru Asada,et al.  Studies on reservoir initialization and dynamics shaping in echo state networks , 2009, ESANN.

[7]  Jochen J. Steil,et al.  Attractor-based computation with reservoirs for online learning of inverse kinematics , 2009, ESANN.

[8]  Benjamin Schrauwen,et al.  On Computational Power and the Order-Chaos Phase Transition in Reservoir Computing , 2008, NIPS.

[9]  Robert A. Legenstein,et al.  A Learning Theory for Reward-Modulated Spike-Timing-Dependent Plasticity with Application to Biofeedback , 2008, PLoS Comput. Biol..

[10]  Marc Schoenauer,et al.  Supervised and Evolutionary Learning of Echo State Networks , 2008, PPSN.

[11]  Benjamin Schrauwen,et al.  Stable Output Feedback in Reservoir Computing Using Ridge Regression , 2008, ICANN.

[12]  R.F. Reinhart,et al.  Recurrent Neural Associative Learning of Forward and Inverse Kinematics for Movement Generation of the Redundant PA-10 Robot , 2008, 2008 ECSIS Symposium on Learning and Adaptive Behaviors for Robotic Systems (LAB-RS).

[13]  Benjamin Schrauwen,et al.  Toward optical signal processing using photonic reservoir computing. , 2008, Optics express.

[14]  Marc Schoenauer,et al.  Unsupervised learning of echo state networks: balancing the double pole , 2008, GECCO '08.

[15]  Benjamin Schrauwen,et al.  Band-pass Reservoir Computing , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[16]  Mahdi Jalili,et al.  Reservoir optimization in recurrent neural networks using kronecker kernels , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[17]  Hendrik Van Brussel,et al.  Pruning and Regularisation in Reservoir Computing: a First Insight , 2008, ESANN.

[18]  Samy Bengio,et al.  Delay learning and polychronization for reservoir computing , 2008, Neurocomputing.

[19]  Jochen J. Steil,et al.  Improving reservoirs using intrinsic plasticity , 2008, Neurocomputing.

[20]  Benjamin Schrauwen,et al.  Compact hardware liquid state machines on FPGA for real-time speech recognition , 2008, Neural Networks.

[21]  Geoffrey E. Hinton.  Reducing the Dimensionality of Data with Neural , 2008 .

[22]  Herbert Jaeger,et al.  Discovering multiscale dynamical features with hierarchical Echo State Networks , 2008 .

[23]  Wolfgang Maass,et al.  Simplified Rules and Theoretical Analysis for Information Bottleneck Optimization and PCA with Spiking Neurons , 2007, NIPS.

[24]  Robert A. Legenstein,et al.  Theoretical Analysis of Learning with Reward-Modulated Spike-Timing-Dependent Plasticity , 2007, NIPS.

[25]  Marc Schoenauer,et al.  Unsupervised Learning of Echo State Networks: A Case Study in Artificial Embryogeny , 2007, Artificial Evolution.

[26]  Benjamin Schrauwen,et al.  The Introduction of Time-Scales in Reservoir Computing, Applied to Isolated Digits Recognition , 2007, ICANN.

[27]  Jiri Pospichal,et al.  Improving the Prediction Accuracy of Echo State Neural Networks by Anti-Oja's Learning , 2007, ICANN.

[28]  Herbert Jaeger,et al.  Echo state network , 2007, Scholarpedia.

[29]  Herbert Jaeger,et al.  Overview of Reservoir Recipes , 2007 .

[30]  Herbert Jaeger,et al.  Optimization and applications of echo state networks with leaky- integrator neurons , 2007, Neural Networks.

[31]  John G. Harris,et al.  Automatic speech recognition using a predictive echo state network classifier , 2007, Neural Networks.

[32]  Mustafa C. Ozturk,et al.  An associative memory readout for ESNs with applications to dynamical pattern recognition , 2007, Neural Networks.

[33]  Simon Haykin,et al.  Decoupled echo state networks with lateral inhibition , 2007, Neural Networks.

[34]  Tadashi Yamazaki,et al.  The cerebellum as a liquid state machine , 2007, Neural Networks.

[35]  Robert A. Legenstein,et al.  2007 Special Issue: Edge of chaos and prediction of computational performance for neural circuit models , 2007 .

[36]  Gordon Pipa,et al.  2007 Special Issue: Fading memory and time series prediction in recurrent networks with different forms of plasticity , 2007 .

[37]  Hendrik Van Brussel,et al.  A first attempt of reservoir pruning for classification problems , 2007, ESANN.

[38]  José Carlos Príncipe,et al.  Special issue on echo state networks and liquid state machines , 2007, Neural Networks.

[39]  Jochen Triesch,et al.  Synergies Between Intrinsic and Synaptic Plasticity Mechanisms , 2007, Neural Computation.

[40]  Jochen J. Steil,et al.  Online reservoir adaptation by intrinsic plasticity for backpropagation-decorrelation and echo state learning , 2007, Neural Networks.

[41]  Benjamin Schrauwen,et al.  An experimental unification of reservoir computing methods , 2007, Neural Networks.

[42]  Ben Jones,et al.  Is there a Liquid State Machine in the Bacterium Escherichia Coli? , 2007, 2007 IEEE Symposium on Artificial Life.

[43]  Min Han,et al.  Support Vector Echo-State Machine for Chaotic Time-Series Prediction , 2007, IEEE Transactions on Neural Networks.

[44]  Jürgen Schmidhuber,et al.  Training Recurrent Networks by Evolino , 2007, Neural Computation.

[45]  Jochen Triesch,et al.  Non-commercial Research and Educational Use including without Limitation Use in Instruction at Your Institution, Sending It to Specific Colleagues That You Know, and Providing a Copy to Your Institution's Administrator. All Other Uses, Reproduction and Distribution, including without Limitation Comm , 2022 .

[46]  Mantas Lukoševičius.  Echo State Networks with Trained Feedbacks , 2007 .

[47]  U. Karmarkar,et al.  Timing in the Absence of Clocks: Encoding Time in Neural Network States , 2007, Neuron.

[48]  Eduardo D. Sontag,et al.  Computational Aspects of Feedback in Neural Circuits , 2006, PLoS Comput. Biol..

[49]  Wolfgang Maass,et al.  A Statistical Analysis of Information- Processing Properties of Lamina-specific Cortical Microcircuit Models , 2022 .

[50]  B. Schölkopf,et al.  Modeling Human Motion Using Binary Latent Variables , 2007 .

[51]  Terrence J. Sejnowski,et al.  What Makes a Dynamical System Computationally Powerful , 2007 .

[52]  Jason Weston,et al.  Scaling Learning Algorithms toward AI , 2007 .

[53]  Thomas Hofmann,et al.  Temporal dynamics of information content carried by neurons in the primary visual cortex , 2007 .

[54]  José Carlos Príncipe,et al.  Analysis and Design of Echo State Networks , 2007, Neural Computation.

[55]  Benjamin Schrauwen,et al.  An overview of reservoir computing: theory, applications and implementations , 2007, ESANN.

[56]  Benjamin Schrauwen,et al.  Adapting reservoirs to get Gaussian distributions , 2007 .

[57]  Yoshua Bengio,et al.  Scaling learning algorithms towards AI , 2007 .

[58]  Jochen J. Steil,et al.  Intrinsic plasticity for reservoir learning algorithms , 2007, ESANN.

[59]  Jochen J. Steil.  Several ways to solve the MSO problem , 2007, ESANN.

[60]  Benjamin Schrauwen,et al.  Linking non-binned spike train kernels to several existing spike train metrics , 2006, ESANN.

[61]  Wei Wang,et al.  The 6th World Congress on Intelligent Control and Automation , 2006 .

[62]  Robert A. Legenstein,et al.  Information Bottleneck Optimization and Independent Component Extraction with Spiking Neurons , 2006, NIPS.

[63]  Danko Nikolic,et al.  Temporal dynamics of information content carried by neurons in the primary visual cortex , 2006, NIPS.

[64]  Geoffrey E. Hinton,et al.  Modeling Human Motion Using Binary Latent Variables , 2006, NIPS.

[65]  Peter Ford Dominey,et al.  A Neurolinguistic Model of Grammatical Construction Processing , 2006, Journal of Cognitive Neuroscience.

[66]  Dan Ventura,et al.  Preparing More Effective Liquid State Machines Using Hebbian Learning , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[67]  Cheng-jian Wei,et al.  Harnessing Non-linearity by Sigmoid-wavelet Hybrid Echo State Networks (SWHESN) , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[68]  András Lörincz,et al.  Critical Echo State Networks , 2006, ICANN.

[69]  Jiri Pospichal,et al.  Merging Echo State and Feedforward Neural Networks for Time Series Forecasting , 2006, ICANN.

[70]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[71]  Benjamin Schrauwen,et al.  Reservoir-based techniques for speech recognition , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[72]  Mantas Lukoševičius,et al.  Time Warping Invariant Echo State Networks , 2006 .

[73]  Peter Michael Young,et al.  A tighter bound for the echo state property , 2006, IEEE Transactions on Neural Networks.

[74]  Jürgen Schmidhuber,et al.  Evolino for recurrent support vector machines , 2005, ESANN.

[75]  Charles W. Anderson,et al.  Exploiting Iso-error Pathways in the N, k-plane to Improve Echo State Network Performance , 2006 .

[76]  Carlos Lourenço.  Dynamical reservoir properties as network effects , 2006, ESANN.

[77]  M. Cernansky,et al.  Feed-forward echo state networks , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[78]  H. Jaeger,et al.  Reservoir riddles: suggestions for echo state network research , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[79]  C. Anderson,et al.  Modeling reward functions for incomplete state representations via echo state networks , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[80]  M. C. Ozturk,et al.  Computing with transiently stable states , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[81]  Dongming Xu,et al.  Direct adaptive control: an echo state network and genetic algorithm approach , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[82]  Eduardo D. Sontag,et al.  Principles of real-time computing with feedback applied to cortical microcircuit models , 2005, NIPS.

[83]  Peter Ford Dominey.  From Sensorimotor Sequence to Grammatical Construction: Evidence from Simulation and Neurophysiology , 2005, Adapt. Behav..

[84]  B. Schrauwen,et al.  Isolated word recognition with the Liquid State Machine: a case study , 2005, Inf. Process. Lett..

[85]  Jochen J. Steil,et al.  Memory in Backpropagation-Decorrelation O(N) Efficient Online Recurrent Learning , 2005, ICANN.

[86]  Jochen Triesch,et al.  A Gradient Rule for the Plasticity of a Neuron's Intrinsic Excitability , 2005, ICANN.

[87]  Julian Eggert,et al.  Short Term Memory and Pattern Matching with Simple Echo State Networks , 2005, ICANN.

[88]  P. Levi,et al.  Meta-Learning for Adaptive Identification of Non-Linear Dynamical Systems , 2005, Proceedings of the 2005 IEEE International Symposium on, Mediterrean Conference on Control and Automation Intelligent Control, 2005..

[89]  W. Gerstner,et al.  Generalized Bienenstock-Cooper-Munro rule for spiking neurons that maximizes information transmission. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[90]  Jochen J. Steil,et al.  Analyzing the weight dynamics of recurrent learning algorithms , 2005, Neurocomputing.

[91]  Daniel Richardson,et al.  Linear Algebra for Time Series of Spikes , 2005, ESANN.

[92]  Benjamin Schrauwen,et al.  Reservoir Computing with Stochastic Bitstream Neurons , 2005 .

[93]  Jochen J. Steil.  Stability of backpropagation-decorrelation efficient O(N) recurrent learning , 2005, ESANN.

[94]  Johannes Schemmel,et al.  Edge of Chaos Computation in Mixed-Mode VLSI - A Hard Liquid , 2004, NIPS.

[95]  Jochen Triesch,et al.  Synergies between Intrinsic and Synaptic Plasticity in Individual Model Neurons , 2004, NIPS.

[96]  Robert A. Legenstein,et al.  Methods for Estimating the Computational Power and Generalization Capability of Neural Microcircuits , 2004, NIPS.

[97]  T. van der Zant,et al.  Identification of motion with echo state network , 2004, Oceans '04 MTS/IEEE Techno-Ocean '04 (IEEE Cat. No.04CH37600).

[98]  J.J. Steil,et al.  Backpropagation-decorrelation: online recurrent learning with O(N) complexity , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[99]  Nils Bertschinger,et al.  Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks , 2004, Neural Computation.

[100]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[101]  G. Miller.  Learning to Forget , 2004, Science.

[102]  Norbert Michael Mayer,et al.  Echo State Networks and Self-Prediction , 2004, BioADIT.

[103]  Marcus Kaiser,et al.  Spatial growth of real-world networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[104]  Peter Ford Dominey.  Complex sensory-motor sequence learning based on recurrent state representation and reinforcement learning , 1995, Biological Cybernetics.

[105]  Henry Markram,et al.  Computational models for generic cortical microcircuits , 2004 .

[106]  Shinji Kusumoto,et al.  Biologically Inspired Approaches to Advanced Information Technology , 2004, Lecture Notes in Computer Science.

[107]  Benjamin Liebald,et al.  Exploration of effects of different network topologies on the ESN signal crosscorrelation matrix spectrum , 2004 .

[108]  Peter Ford Dominey,et al.  Identification of prosodic attitudes by a temporal recurrent network. , 2003, Brain research. Cognitive brain research.

[109]  Chrisantha Fernando,et al.  Pattern Recognition in a Bucket , 2003, ECAL.

[110]  Peter Ford Dominey,et al.  Neurological basis of language and sequential cognition: Evidence from simulation, aphasia, and ERP studies , 2003, Brain and Language.

[111]  Robert M. French,et al.  Catastrophic interference in connectionist networks , 2003 .

[112]  Henry Markram,et al.  Computer models and analysis tools for neural microcircuits , 2003 .

[113]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[114]  Chris I. De Zeeuw,et al.  Dynamical Working Memory and Timed Responses: The Role of Reverberating Loops in the Olivo-Cerebellar System , 2002, Neural Computation.

[115]  Danil V. Prokhorov,et al.  Adaptive behavior with fixed weights in RNN: an overview , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[116]  D. Munday.  Edge of chaos. , 2002, Journal of the Royal Society of Medicine.

[117]  Herbert Jaeger,et al.  Adaptive Nonlinear System Identification with Echo State Networks , 2002, NIPS.

[118]  Henry Markram,et al.  A Model for Real-Time Computation in Generic Neural Microcircuits , 2002, NIPS.

[119]  D. Wolpert.  The Supervised Learning No-Free-Lunch Theorems , 2002 .

[120]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[121]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[122]  Amir F. Atiya,et al.  New results on recurrent network training: unifying the algorithms and accelerating convergence , 2000, IEEE Trans. Neural Networks Learn. Syst..

[123]  Naftali Tishby,et al.  The information bottleneck method , 2000, ArXiv.

[124]  Peter Ford Dominey,et al.  Neural network processing of natural language: I. Sensitivity to serial, temporal and abstract structure of language in the infant , 2000 .

[125]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[126]  G B Stanley,et al.  Reconstruction of Natural Scenes from Ensemble Responses in the Lateral Geniculate Nucleus , 1999, The Journal of Neuroscience.

[127]  Tafsir Thiam,et al.  The Boltzmann machine , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[128]  Christof Koch,et al.  How voltage-dependent conductances can adapt to maximize the information encoded by neuronal firing rate , 1999, Nature Neuroscience.

[129]  B. Farhang-Boroujeny,et al.  Adaptive Filters: Theory and Applications , 1999 .

[130]  Emile H. L. Aarts,et al.  Boltzmann machines , 1998 .

[131]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[132]  H. Markram,et al.  Differential signaling via the same axon of neocortical pyramidal neurons. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[133]  Chuanyi Ji,et al.  Fast training of recurrent networks based on the EM algorithm , 1998, IEEE Trans. Neural Networks.

[134]  L. Abbott,et al.  Responses of neurons in primary and inferior temporal visual cortices to natural scenes , 1997, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[135]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[136]  Yoshua Bengio,et al.  Hierarchical Recurrent Neural Networks for Long-Term Dependencies , 1995, NIPS.

[137]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[138]  M M Merzenich,et al.  Temporal information transformed into a spatial code by a neural network with realistic properties , 1995, Science.

[139]  Lee A. Feldkamp,et al.  Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks , 1994, IEEE Trans. Neural Networks.

[140]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[141]  James P. Crutchfield,et al.  Dynamics, computation, and the “edge of chaos”: a re-examination , 1993, adap-org/9306003.

[142]  Yuichi Nakamura,et al.  Approximation of dynamical systems by continuous time recurrent neural networks , 1993, Neural Networks.

[143]  K. Doya,et al.  Bifurcations in the learning of recurrent neural networks , 1992, [Proceedings] 1992 IEEE International Symposium on Circuits and Systems.

[144]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[145]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[146]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[147]  M. V. Rossum,et al.  In Neural Computation , 2022 .

[148]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[149]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[150]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[151]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[152]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[153]  W. Kautz.  Transient synthesis in the time domain , 1954 .

[154]  J. Knott.  The organization of behavior: A neuropsychological theory , 1951 .

[155]  D. Hebb.  The Organization of behavior : a neuropsychological theory / D.O. Hebb , 1949 .

[156]  W. Maass,et al.  What makes a dynamical system computationally powerful ? , 2022 .