Recent advances in efficient learning of recurrent networks

Recurrent neural networks (RNNs) carry the promise of implementing efficient and biologically plausible signal processing. They both are optimally suited for a wide area of applications when dealing with spatiotemporal data or causalities and provide explanation of cog- nitive phenomena of the human brain. Recently, a few new fundamental paradigms connected to RNNs have been developed which allow insights into their potential for information processing. They also pave the way towards new efficient training algorithms which overcome the well-known problem of long-term dependencies. This tutorial gives an overview of this recent developments in efficient, biologically plausible recurrent informa- tion processing.

[1]  Claudius Gros,et al.  Stimulus processing and unsupervised learning in autonomously active recurrent networks , 2009, ESANN.

[2]  Zhigang Zeng,et al.  Analysis and Design of Associative Memories Based on Recurrent Neural Networks with Linear Saturation Activation Functions and Time-Varying Delays , 2007, Neural Computation.

[3]  Chih-Min Lin,et al.  Adaptive recurrent cerebellar model articulation controller for linear ultrasonic motor with optimal learning rates , 2007, Neurocomputing.

[4]  Chris Eliasmith,et al.  Solving the Problem of Negative Synaptic Weights in Cortical Models , 2008, Neural Computation.

[5]  Nicholas G. Maratos,et al.  A Nonfeasible Gradient Projection Recurrent Neural Network for Equality-Constrained Optimization Problems , 2008, IEEE Transactions on Neural Networks.

[6]  Giovanni Costantini,et al.  Quasi-Lagrangian Neural Network for Convex Quadratic Optimization , 2008, IEEE Transactions on Neural Networks.

[7]  Liu Sheng,et al.  A Normalized Adaptive Training of Recurrent Neural Networks With Augmented Error Gradient , 2008, IEEE Transactions on Neural Networks.

[8]  Luís A. Alexandre,et al.  Reservoir computing for static pattern recognition , 2009, ESANN.

[9]  Jinde Cao,et al.  Stability in static delayed neural networks: A nonlinear measure approach , 2006, Neurocomputing.

[10]  Jun Tani,et al.  Developmental learning of complex syntactical song in the Bengalese finch: A neural network model , 2008, Neural Networks.

[11]  Frank-Michael Schleif,et al.  Advances in computational intelligence and learning , 2010 .

[12]  Xiaolin Hu,et al.  Solving Generally Constrained Generalized Linear Variational Inequalities Using the General Projection Neural Networks , 2007, IEEE Transactions on Neural Networks.

[13]  Alexei A. Koulakov,et al.  Neural Integrator: A Sandpile Model , 2006, Neural Computation.

[14]  Herbert Jaeger,et al.  Optimization and applications of echo state networks with leaky- integrator neurons , 2007, Neural Networks.

[15]  Danil V. Prokhorov Training Recurrent Neurocontrollers for Robustness With Derivative-Free Kalman Filter , 2006, IEEE Transactions on Neural Networks.

[16]  Richard Kempter,et al.  Memory Capacity for Sequences in a Recurrent Network with Biological Constraints , 2006, Neural Computation.

[17]  Mathukumalli Vidyasagar,et al.  A Theory of Learning and Generalization , 1997 .

[18]  Luc De Raedt,et al.  Ranking neurons for mining structure-activity relations in biological neural networks: NeuronRank , 2007, Neurocomputing.

[19]  Alexandre Pouget,et al.  Dynamical Constraints on Using Precise Spike Timing to Compute in Recurrent Cortical Networks , 2008, Neural Computation.

[20]  Gordon Pipa,et al.  2007 Special Issue: Fading memory and time series prediction in recurrent networks with different forms of plasticity , 2007 .

[21]  Marco Wiering,et al.  Recurrent neural network modeling of nearshore sandbar behavior , 2007, Neural Networks.

[22]  Danil V. Prokhorov Training Recurrent Neurocontrollers for Real-Time Applications , 2007, IEEE Transactions on Neural Networks.

[23]  Mohamed S. Kamel,et al.  A Cooperative Recurrent Neural Network for Solving L1 Estimation Problems with General Linear Constraints , 2008, Neural Computation.

[24]  Barbara Hammer,et al.  Generalization Ability of Folding Networks , 2001, IEEE Trans. Knowl. Data Eng..

[25]  Frank Emmert-Streib Influence of the neural network topology on the learning dynamics , 2006, Neurocomputing.

[26]  K. Patan Stability Analysis and the Stabilization of a Class of Discrete-Time Dynamic Neural Networks , 2007, IEEE Transactions on Neural Networks.

[27]  Nicolas Brunel,et al.  How Noise Affects the Synchronization Properties of Recurrent Networks of Inhibitory Neurons , 2006, Neural Computation.

[28]  Wangqiang Niu,et al.  Recurrent network simulations of two types of non-concentric retinal ganglion cells , 2007, Neurocomputing.

[29]  Long Cheng,et al.  A Recurrent Neural Network for Hierarchical Control of Interconnected Dynamic Systems , 2007, IEEE Transactions on Neural Networks.

[30]  Michael E. Hasselmo,et al.  Reversed and forward buffering of behavioral spike sequences enables retrospective and prospective retrieval in hippocampal regions CA3 and CA1 , 2008, Neural Networks.

[31]  Raymond H. Cuijpers,et al.  Generalisation of action sequences in RNNPB networks with mirror properties , 2009, ESANN 2009.

[32]  Zhigang Zeng,et al.  Global exponential stability of recurrent neural networks with time-varying delays in the presence of strong external stimuli , 2006, Neural Networks.

[33]  De RaedtLuc,et al.  Ranking neurons for mining structure-activity relations in biological neural networks , 2007 .

[34]  Yeng Chai Soh,et al.  Robust Adaptive Gradient-Descent Training Algorithm for Recurrent Neural Networks in Discrete Time Domain , 2008, IEEE Transactions on Neural Networks.

[35]  André Grüning,et al.  Elman Backpropagation as Reinforcement for Simple Recurrent Networks , 2007, Neural Computation.

[36]  Adam Ponzi,et al.  Dynamical model of salience gated working memory, action selection and reinforcement based on basal ganglia and dopamine feedback , 2008, Neural Networks.

[37]  Tianping Chen,et al.  Global $\mu$ -Stability of Delayed Neural Networks With Unbounded Time-Varying Delays , 2007, IEEE Transactions on Neural Networks.

[38]  Chia-Feng Juang,et al.  Recurrent fuzzy network design using hybrid evolutionary learning algorithms , 2007, Neurocomputing.

[39]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[40]  Garrison W. Cottrell,et al.  2007 Special Issue: Learning grammatical structure with Echo State Networks , 2007 .

[41]  Hanyong Shao,et al.  Delay-Dependent Stability for Recurrent Neural Networks With Time-Varying Delays , 2008, IEEE Transactions on Neural Networks.

[42]  Heiko Wersing Learning Lateral Interactions for Feature Binding and Sensory Segmentation , 2001, NIPS.

[43]  Jun Tani,et al.  A model for learning to segment temporal sequences, utilizing a mixture of RNN experts together with adaptive variance , 2007, Neural Networks.

[44]  Chia-Feng Juang,et al.  Hierarchical Singleton-Type Recurrent Neural Fuzzy Networks for Noisy Speech Recognition , 2007, IEEE Transactions on Neural Networks.

[45]  Benjamin Schrauwen,et al.  Toward optical signal processing using photonic reservoir computing. , 2008, Optics express.

[46]  Filip Piekniewski,et al.  Spontaneous scale-free structure of spike flow graphs in recurrent neural networks , 2008, Neural Networks.

[47]  Zhi-Hong Mao,et al.  Dynamics of Winner-Take-All Competition in Recurrent Neural Networks With Lateral Inhibition , 2007, IEEE Transactions on Neural Networks.

[48]  Danilo P. Mandic,et al.  An Augmented Extended Kalman Filter Algorithm for Complex-Valued Recurrent Neural Networks , 2006, Neural Computation.

[49]  Alessio Micheli,et al.  Adaptive Contextual Processing of Structured Data by Recursive Neural Networks: A Survey of Computational Properties , 2007, Perspectives of Neural-Symbolic Integration.

[50]  Yukio-Pegio Gunji,et al.  Recurrent neural network architecture with pre-synaptic inhibition for incremental learning , 2006, Neural Networks.

[51]  Nian Zhang,et al.  Time series prediction with recurrent neural networks trained by a hybrid PSO-EA algorithm , 2004, Neurocomputing.

[52]  Peter Tiño,et al.  Learning Beyond Finite Memory in Recurrent Networks of Spiking Neurons , 2005, Neural Computation.

[53]  Peter Michael Young,et al.  A tighter bound for the echo state property , 2006, IEEE Transactions on Neural Networks.

[54]  Qingshan Liu,et al.  Two k-winners-take-all networks with discontinuous activation functions , 2008, Neural Networks.

[55]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[56]  Johan A. K. Suykens,et al.  Robust local stability of multilayer recurrent neural networks , 2000, IEEE Trans. Neural Networks Learn. Syst..

[57]  Dimitris K. Tasoulis,et al.  Generalized locally recurrent probabilistic neural networks with application to text-independent speaker verification , 2007, Neurocomputing.

[58]  Yutaka Sakai,et al.  Computational algorithms and neuronal network models underlying decision processes , 2006, Neural Networks.

[59]  Jun Wang,et al.  Noise-Induced Stabilization of the Recurrent Neural Networks With Mixed Time-Varying Delays and Markovian-Switching Parameters , 2007, IEEE Transactions on Neural Networks.

[60]  Ernst Niebur,et al.  Exact Solutions for Rate and Synchrony in Recurrent Networks of Coincidence Detectors , 2008, Neural Computation.

[61]  Mark C. W. van Rossum,et al.  Adaptive Integration in the Visual Cortex by Depressing Recurrent Cortical Circuits , 2008, Neural Computation.

[62]  Tadashi Yamazaki,et al.  The cerebellum as a liquid state machine , 2007, Neural Networks.

[63]  Michael Margaliot,et al.  A New Approach to Knowledge-Based Design of Recurrent Neural Networks , 2008, IEEE Transactions on Neural Networks.

[64]  Herbert Jaeger,et al.  Overview of Reservoir Recipes , 2007 .

[65]  Zhidong Deng,et al.  Collective Behavior of a Small-World Recurrent Neural System With Scale-Free Distribution , 2007, IEEE Transactions on Neural Networks.

[66]  Maria Teresinha Arns Steiner,et al.  A new approach to solve the traveling salesman problem , 2007, Neurocomputing.

[67]  Ganesh K. Venayagamoorthy,et al.  Online design of an echo state network based wide area monitor for a multimachine power system , 2007, Neural Networks.

[68]  Shubao Liu,et al.  A Simplified Dual Neural Network for Quadratic Programming With Its KWTA Application , 2006, IEEE Transactions on Neural Networks.

[69]  Alexander A. Frolov,et al.  Boolean Factor Analysis by Attractor Neural Network , 2007, IEEE Transactions on Neural Networks.

[70]  Xiao-Jing Wang,et al.  Mean-Driven and Fluctuation-Driven Persistent Activity in Recurrent Networks , 2007, Neural Computation.

[71]  Martin Rehn,et al.  Tonically driven and self-sustaining activity in the lamprey hemicord: When can they co-exist? , 2007, Neurocomputing.

[72]  Jinde Cao,et al.  Stability and Hopf Bifurcation of a General Delayed Recurrent Neural Network , 2008, IEEE Transactions on Neural Networks.

[73]  Wei-Po Lee,et al.  A clustering-based approach for inferring recurrent neural networks as gene regulatory networks , 2008, Neurocomputing.

[74]  Xiaolin Hu,et al.  An Improved Dual Neural Network for Solving a Class of Quadratic Programming Problems and Its $k$-Winners-Take-All Application , 2008, IEEE Transactions on Neural Networks.

[75]  Hendrik Van Brussel,et al.  Non-markovian process modelling with Echo State Networks , 2009, ESANN.

[76]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[77]  Lubica Benusková,et al.  Organization of the state space of a simple recurrent network before and after training on recursive linguistic structures , 2007, Neural Networks.

[78]  Stefan Rotter,et al.  Correlations and Population Dynamics in Cortical Networks , 2008, Neural Computation.

[79]  H. Sebastian Seung,et al.  Neural Voting Machines , 2006, Neural Networks.

[80]  Herbert Jaeger,et al.  Discovering multiscale dynamical features with hierarchical Echo State Networks , 2008 .

[81]  Zhigang Zeng,et al.  Improved conditions for global exponential stability of recurrent neural networks with time-varying delays , 2006, IEEE Transactions on Neural Networks.

[82]  Erol Gelenbe,et al.  Random Neural Networks with Synchronized Interactions , 2008, Neural Computation.

[83]  Arvind Kumar,et al.  Emergence of population synchrony in a layered network of the cat visual cortex , 2007, Neurocomputing.

[84]  Byung Ro Moon,et al.  A Hybrid Neurogenetic Approach for Stock Forecasting , 2007, IEEE Transactions on Neural Networks.

[85]  Kay Chen Tan,et al.  Dynamics analysis and analog associative memory of networks with LT neurons , 2006, IEEE Transactions on Neural Networks.

[86]  Min Han,et al.  Support Vector Echo-State Machine for Chaotic Time-Series Prediction , 2007, IEEE Transactions on Neural Networks.

[87]  Emilio Salinas,et al.  When Response Variability Increases Neural Network Robustness to Synaptic Noise , 2005, Neural Computation.

[88]  Qing-Long Han,et al.  Discrete-Time Analogs for a Class of Continuous-Time Recurrent Neural Networks , 2007, IEEE Transactions on Neural Networks.

[89]  Huaguang Zhang,et al.  Global Asymptotic Stability of Recurrent Neural Networks With Multiple Time-Varying Delays , 2008, IEEE Transactions on Neural Networks.

[90]  Danil V. Prokhorov,et al.  Toyota Prius HEV neurocontrol and diagnostics , 2008, Neural Networks.

[91]  Jochen J. Steil,et al.  Input space bifurcation manifolds of recurrent neural networks , 2005, Neurocomputing.

[92]  Mia Setic,et al.  A model of the illusory contour formation based on dendritic computation , 2007, Neurocomputing.

[93]  Ioannis B. Theocharis,et al.  Locally recurrent neural networks for long-term wind speed and power prediction , 2006, Neurocomputing.

[94]  Youxian Sun,et al.  An estimation of the domain of attraction for recurrent neural networks with time-varying delays , 2008, Neurocomputing.

[95]  Kay Chen Tan,et al.  An asynchronous recurrent linear threshold network approach to solving the traveling salesman problem , 2008, Neurocomputing.

[96]  Jun Wang,et al.  An Improved Algebraic Criterion for Global Exponential Stability of Recurrent Neural Networks With Time-Varying Delays , 2008, IEEE Transactions on Neural Networks.

[97]  Donald C. Wunsch,et al.  Training Winner-Take-All Simultaneous Recurrent Neural Networks , 2007, IEEE Transactions on Neural Networks.

[98]  Jochen J. Steil,et al.  Attractor-based computation with reservoirs for online learning of inverse kinematics , 2009, ESANN.

[99]  Wen Yu,et al.  Nonlinear system identification with recurrent neural networks and dead-zone Kalman filter algorithm , 2007, Neurocomputing.

[100]  Jun Tani,et al.  Dynamic and interactive generation of object handling behaviors by a small humanoid robot using a dynamic neural network model , 2006, Neural Networks.

[101]  Ioannis B. Theocharis,et al.  A locally recurrent fuzzy neural network with application to the wind speed prediction using spatial correlation , 2007, Neurocomputing.

[102]  András Lörincz,et al.  PIRANHA: Policy iteration for recurrent artificial neural networks with hidden activities , 2006, Neurocomputing.

[103]  J. Michael Herrmann,et al.  Criticality of avalanche dynamics in adaptive recurrent networks , 2006, Neurocomputing.

[104]  Ivan Tyukin,et al.  Adaptive Classification of Temporal Signals in Fixed-Weight Recurrent Neural Networks: An Existence Proof , 2007, Neural Computation.

[105]  J.J. Steil,et al.  Backpropagation-decorrelation: online recurrent learning with O(N) complexity , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[106]  Alessandro Treves,et al.  Modelling adaptation aftereffects in associative memory , 2007, Neurocomputing.

[107]  Alessio Micheli,et al.  Self-Organizing Maps for Time Series , 2005 .

[108]  Aluizio F. R. Araújo,et al.  A Taxonomy for Spatiotemporal Connectionist Networks Revisited: The Unsupervised Case , 2003, Neural Computation.

[109]  Igor Farkas,et al.  Syntactic systematicity in sentence processing with a recurrent self-organizing network , 2008, Neurocomputing.

[110]  Zhang Yi,et al.  Multiperiodicity and Attractivity of Delayed Recurrent Neural Networks With Unsaturating Piecewise Linear Transfer Functions , 2008, IEEE Transactions on Neural Networks.

[111]  Qingshan Liu,et al.  A One-Layer Recurrent Neural Network With a Discontinuous Hard-Limiting Activation Function for Quadratic Programming , 2008, IEEE Transactions on Neural Networks.

[112]  Xuyang Lou,et al.  Delay-Dependent Criteria for Global Robust Periodicity of Uncertain Switched Recurrent Neural Networks With Time-Varying Delay , 2008, IEEE Transactions on Neural Networks.

[113]  Wen Yu,et al.  Multiple recurrent neural networks for stable adaptive control , 2006, Neurocomputing.

[114]  Kazuyuki Aihara,et al.  Selectivity and Stability via Dendritic Nonlinearity , 2007, Neural Computation.

[115]  Peter Tiño,et al.  Markovian Bias of Neural-based Architectures With Feedback Connections , 2007, Perspectives of Neural-Symbolic Integration.

[116]  Randall D. Beer,et al.  Parameter Space Structure of Continuous-Time Recurrent Neural Networks , 2006, Neural Computation.

[117]  Chia-Feng Juang,et al.  Birdsong recognition using prediction-based recurrent neural fuzzy networks , 2007, Neurocomputing.

[118]  Shan Liu,et al.  Development of a new EDRNN procedure in control of human arm trajectories , 2008, Neurocomputing.

[119]  Jochen J. Steil,et al.  Input output stability of recurrent neural networks , 1999 .

[120]  Minoru Asada,et al.  Studies on reservoir initialization and dynamics shaping in echo state networks , 2009, ESANN.

[121]  Donald C. Wunsch,et al.  Modeling of gene regulatory networks with hybrid differential evolution and particle swarm optimization , 2007, Neural Networks.

[122]  Benjamin Schrauwen,et al.  Event detection and localization for small mobile robots using reservoir computing , 2008, Neural Networks.

[123]  Shumin Fei,et al.  Exponential state estimation for recurrent neural networks with distributed delays , 2007, Neurocomputing.

[124]  Henrik Jacobsson,et al.  The Crystallizing Substochastic Sequential Machine Extractor: CrySSMEx , 2006, Neural Computation.

[125]  Jun Tani,et al.  Emergence of Functional Hierarchy in a Multiple Timescale Neural Network Model: A Humanoid Robot Experiment , 2008, PLoS Comput. Biol..

[126]  Helge J. Ritter,et al.  Gestalt-based action segmentation for robot task learning , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[127]  Antonio Fernández Caballero,et al.  Parametric improvement of lateral interaction in accumulative computation in motion-based segmentation , 2008 .

[128]  Xiaolin Hu,et al.  Solving Pseudomonotone Variational Inequalities and Pseudoconvex Optimization Problems Using the Projection Neural Network , 2006, IEEE Transactions on Neural Networks.

[129]  Chia-Feng Juang,et al.  Mold temperature control of a rubber injection-molding machine by TSK-type recurrent neural fuzzy network , 2006, Neurocomputing.

[130]  Benjamin Schrauwen,et al.  An experimental unification of reservoir computing methods , 2007, Neural Networks.

[131]  Franco Scarselli,et al.  Recursive processing of cyclic graphs , 2006, IEEE Transactions on Neural Networks.

[132]  Jin Bae Park,et al.  Adaptive Output Feedback Control of Flexible-Joint Robots Using Neural Networks: Dynamic Surface Design Approach , 2008, IEEE Transactions on Neural Networks.

[133]  Zhigang Zeng,et al.  Positive invariant and global exponential attractive sets of neural networks with time-varying delays , 2008, Neurocomputing.

[134]  Benjamin Schrauwen,et al.  On Computational Power and the Order-Chaos Phase Transition in Reservoir Computing , 2008, NIPS.

[135]  Jun Wang,et al.  A Novel Recurrent Neural Network for Solving Nonlinear Optimization Problems With Inequality Constraints , 2008, IEEE Transactions on Neural Networks.

[136]  Katsunari Shibata,et al.  A model to explain the emergence of reward expectancy neurons using reinforcement learning and neural network , 2006, Neurocomputing.

[137]  Benjamin Schrauwen,et al.  Compact hardware liquid state machines on FPGA for real-time speech recognition , 2008, Neural Networks.

[138]  Jun Wang,et al.  Global exponential periodicity and global exponential stability of a class of recurrent neural networks with various activation functions and time-varying delays , 2007, Neural Networks.

[139]  Martin T. Hagan,et al.  Backpropagation Algorithms for a Broad Class of Dynamic Networks , 2007, IEEE Transactions on Neural Networks.

[140]  Meiqin Liu,et al.  Delayed Standard Neural Network Models for Control Systems , 2007, IEEE Transactions on Neural Networks.

[141]  Zhang Yi,et al.  Switching analysis of 2-D neural networks with nonsaturating linear threshold transfer functions , 2008, Neurocomputing.

[142]  Jürgen Schmidhuber,et al.  Training Recurrent Networks by Evolino , 2007, Neural Computation.

[143]  Arvind Kumar,et al.  The High-Conductance State of Cortical Networks , 2008, Neural Computation.

[144]  Elisa Ricci,et al.  Analog neural network for support vector machine learning , 2006, IEEE Transactions on Neural Networks.

[145]  Chih-Min Lin,et al.  RCMAC Hybrid Control for MIMO Uncertain Nonlinear Systems Using Sliding-Mode Technology , 2007, IEEE Transactions on Neural Networks.

[146]  Bo Cartling,et al.  On the implicit acquisition of a context-free grammar by a simple recurrent neural network , 2008, Neurocomputing.

[147]  Qiankun Song,et al.  Exponential stability of recurrent neural networks with both time-varying delays and general activation functions via LMI approach , 2008, Neurocomputing.

[148]  Jinde Cao,et al.  Global Asymptotical Stability of Recurrent Neural Networks With Multiple Discrete Delays and Distributed Delays , 2006, IEEE Transactions on Neural Networks.

[149]  Gang Feng,et al.  A new neural network for solving nonlinear projection equations , 2007, Neural Networks.

[150]  R.F. Reinhart,et al.  Recurrent Neural Associative Learning of Forward and Inverse Kinematics for Movement Generation of the Redundant PA-10 Robot , 2008, 2008 ECSIS Symposium on Learning and Adaptive Behaviors for Robotic Systems (LAB-RS).

[151]  Christof Cebulla,et al.  Asymptotic Behavior and Synchronizability Characteristics of a Class of Recurrent Neural Networks , 2007, Neural Computation.

[152]  Jochen J. Steil,et al.  Online stability of backpropagation-decorrelation recurrent learning , 2006, Neurocomputing.

[153]  Peter Dayan,et al.  Encoding and Decoding Spikes for Dynamic Stimuli , 2008, Neural Computation.

[154]  Qingshan Liu,et al.  A One-Layer Recurrent Neural Network with a Discontinuous Activation Function for Linear Programming , 2008, Neural Computation.

[155]  Douglas C. Hittle,et al.  Robust Reinforcement Learning Control Using Integral Quadratic Constraints for Recurrent Neural Networks , 2007, IEEE Transactions on Neural Networks.

[156]  Zidong Wang,et al.  Global exponential stability of generalized recurrent neural networks with discrete and distributed delays , 2006, Neural Networks.

[157]  Zhenzhen Liu,et al.  A Fast and Scalable Recurrent Neural Network Based on Stochastic Meta Descent , 2008, IEEE Transactions on Neural Networks.

[158]  Bo Gao,et al.  Equilibria and Their Bifurcations in a Recurrent Neural Network Involving Iterates of a Transcendental Function , 2008, IEEE Transactions on Neural Networks.

[159]  Frédéric Henry,et al.  Temporal pattern identification using spike-timing dependent plasticity , 2007, Neurocomputing.

[160]  Jochen J. Steil,et al.  Improving reservoirs using intrinsic plasticity , 2008, Neurocomputing.

[161]  Yun Zou,et al.  Improved delay-dependent exponential stability criteria for discrete-time recurrent neural networks with time-varying delays , 2008, Neurocomputing.

[162]  Fionn Murtagh,et al.  Wavelet-based nonlinear multiscale decomposition model for electricity load forecasting , 2006, Neurocomputing.