Minimal Echo State Networks for Visualisation

We create an artificial neural network which is a version of echo state machines, ESNs. ESNs are recurrent neural networks but unlike most recurrent networks, they come with an efficient training method. We have previously \cite{fyfe:douglas1_2011} adapted this method using ideas from neuroscale \cite{tipping: thesis} so that the network is optimal for projecting multivariate time series data onto a low dimensional manifold so that the structure in the time series can be identified by eye. In this paper, we investigate a minimal architecture echo state machine in the context of visualisation and show that it does not perform as well as the original. Using a financial time series, we investigate 3 methods for regaining the power of the standard echo state machine.

[1]  Colin Fyfe,et al.  Extending Metric Multidimensional Scaling with Bregman Divergences , 2010, IEA/AIE.

[2]  Michael E. Tipping,et al.  Shadow targets: A novel algorithm for topographic projections by radial basis functions , 1998, Neurocomputing.

[3]  Benjamin Liebald,et al.  Exploration of effects of different network topologies on the ESN signal crosscorrelation matrix spectrum , 2004 .

[4]  C. Fyfe,et al.  Initializing Echo State Networks with topographic maps , 2011 .

[5]  Colin Fyfe,et al.  Two topographic maps for data visualisation , 2007, Data Mining and Knowledge Discovery.

[6]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[7]  C Fyfe,et al.  A scale-invariant feature map. , 1996, Network.

[8]  Mantas Lukoševičius,et al.  On self-organizing reservoirs and their hierarchies , 2010 .

[9]  Michael E. Tipping.  Topographic mappings and feed-forward neural networks , 1996 .

[10]  Tzai-Der Wang,et al.  Training Echo State Networks with Neuroscale , 2011, 2011 International Conference on Technologies and Applications of Artificial Intelligence.

[11]  Michael E. Tipping,et al.  Shadow targets: a novel algorithm for topographic projections by radial basis functions , 1997 .

[12]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[13]  R. Shiller.  Irrational Exuberance Ed. 2 , 2005 .

[14]  Christopher M. Bishop,et al.  GTM: The Generative Topographic Mapping , 1998, Neural Computation.

[15]  Peter Tiño,et al.  Minimum Complexity Echo State Network , 2011, IEEE Transactions on Neural Networks.

[16]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..