Multitasking attractor networks with neuronal threshold noise

We consider the multitasking associative network in the low-storage limit and we study its phase diagram with respect to the noise level T and the degree d of dilution in pattern entries. We find that the system is characterized by a rich variety of stable states, including pure states, parallel retrieval states, hierarchically organized states and symmetric mixtures (remarkably, both even and odd), whose complexity increases as the number of patterns P grows. The analysis is performed both analytically and numerically: Exploiting techniques based on partial differential equations, we are able to get the self-consistencies for the order parameters. Such self-consistency equations are then solved and the solutions are further checked through stability theory to catalog their organizations into the phase diagram, which is outlined at the end. This is a further step towards the understanding of spontaneous parallel processing in associative networks.

[1]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[2]  Peter Sollich,et al.  Theory of Neural Information Processing Systems , 2005 .

[3]  Sompolinsky,et al.  Neural networks with nonlinear synapses and a static noise. , 1986, Physical review. A, General physics.

[4]  D. M. Titterington,et al.  Neural Networks: A Review from a Statistical Perspective , 1994 .

[5]  A.C.C. Coolen,et al.  Replicated transfer matrix analysis of Ising spin models on 'small world' lattices , 2004 .

[6]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[7]  A. Barra,et al.  A thermodynamic perspective of immune capabilities. , 2011, Journal of theoretical biology.

[8]  B. Wemmenhove,et al.  Finite connectivity attractor neural networks , 2003 .

[9]  Jeffrey L. Krichmar,et al.  Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines , 2001, Complex..

[10]  Elena Agliari,et al.  Multitasking associative networks. , 2011, Physical review letters.

[11]  F. Y. Wu.  The Potts model , 1982 .

[12]  A. Barra,et al.  A mechanical approach to mean field spin models , 2008, 0812.1978.

[13]  Elena Agliari,et al.  Parallel retrieval of correlated patterns: From Hopfield networks to Boltzmann machines , 2013, Neural Networks.

[14]  Efraim Turban,et al.  Neural Networks in Finance and Investing: Using Artificial Intelligence to Improve Real-World Performance , 1992 .

[15]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[16]  Adriano Barra,et al.  On the equivalence of Hopfield networks and Boltzmann Machines , 2011, Neural Networks.

[17]  C. Malsburg,et al.  How patterned neural connections can be set up by self-organization , 1976, Proceedings of the Royal Society of London. Series B. Biological Sciences.