Short term memory in input-driven linear dynamical systems

We investigate the relation between two quantitative measures characterizing short term memory in input driven dynamical systems, namely the short term memory capacity (MC) [3] and the Fisher memory curve (FMC) [2]. We show that even though MC and FMC map the memory structure of the system under investigation from two quite different perspectives, for linear input driven dynamical systems they are in fact closely related. In particular, under some assumptions, the two quantities can be interpreted as squared 'Mahalanobis' norms of images of the input vector under the system's dynamics. We also offer a detailed rigorous analysis of the relation between MC and FMC in cases of symmetric and cyclic dynamic couplings.

[1]  Xueyan Tang,et al.  Optimizing static job scheduling in a network of heterogeneous computers , 2000, Proceedings 2000 International Conference on Parallel Processing.

[2]  David D. Yao,et al.  Optimal load balancing and scheduling in a distributed computer system , 1991, JACM.

[3]  Kevin Li.  Optimizing Average Job Response Time via Decentralized Probabilistic Job Dispatching in Heterogeneous Multiple Computer Systems , 1998, Comput. J..

[4]  Keqin Li,et al.  Optimal load distribution in nondedicated heterogeneous cluster and grid computing environments , 2008, J. Syst. Archit..

[5]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[6]  Garrison W. Cottrell,et al.  2007 Special Issue: Learning grammatical structure with Echo State Networks , 2007 .

[7]  Asser N. Tantawi,et al.  Optimal static load balancing in distributed computer systems , 1985, JACM.

[8]  Herbert Jaeger,et al.  Echo state network , 2007, Scholarpedia.

[9]  Jie Li,et al.  Optimal Load Balancing in Distributed Computer Systems , 1997 .

[10]  Surya Ganguli,et al.  Memory traces in dynamical systems , 2008, Proceedings of the National Academy of Sciences.

[11]  C. Anderson,et al.  Modeling reward functions for incomplete state representations via echo state networks , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[12]  John G. Harris,et al.  Minimum mean squared error time series classification using an echo state network prediction model , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[13]  Anurag Kumar,et al.  Adaptive Optimal Load Balancing in a Nonhomogeneous Multiserver System with a Central Job Scheduler , 1990, IEEE Trans. Computers.

[14]  Keqin Li.  Minimizing mean response time in heterogeneous multiple computer systems with a central stochastic job dispatcher , 1998 .

[15]  Keqin Li.  Minimizing the probability of load imbalance in heterogeneous distributed computer systems , 2002 .

[16]  C. G. Rommen.  The probability of load balancing success in a homogeneous network , 1991 .

[17]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[18]  Ali R. Hurson,et al.  Scheduling and Load Balancing in Parallel and Distributed Systems , 1995 .

[19]  Stephen A. Jarvis,et al.  Allocating non-real-time and soft real-time jobs in multiclusters , 2006, IEEE Transactions on Parallel and Distributed Systems.

[20]  Peter Tiño,et al.  Minimum Complexity Echo State Network , 2011, IEEE Transactions on Neural Networks.