Rotor Associative Memory with a Periodic Activation Function

Complex-valued Associative Memory (CAM) can store multi-state patterns unlike Hopfield Associative Memory (HAM). CAM stores not only given training patterns but also many spurious patterns, such as their rotated patterns, at the same time. Rotor Associative Memory (RAM) can make the most of rotated patterns unstable but the reversed patterns remain stable. In the present work, we propose RAM with a Periodic Activation Function (PAF) to make the reversed patterns unstable. PAF is an activation function that Aizenberg introduced to CAM. We prove that RAM with a PAF has far fewer spurious patterns by using dynamic associative memories which can search the stored patterns.

[1]  Jacek M. Zurada,et al.  Complex-valued multistate neural associative memory , 1996, IEEE Trans. Neural Networks.

[2]  André J. Noest,et al.  Phasor Neural Networks , 1987, NIPS.

[3]  Masaki Kobayashi,et al.  Reducing Spurious States by Rotor Associative Memory , 2011 .

[4]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[5]  H. Aoki.  Image Association Using a Complex-Valued Associative Memory Model , 2000 .

[6]  Joos Vandewalle,et al.  Multi-Valued and Universal Binary Neurons: Theory, Learning and Applications , 2012 .

[7]  Masaki Kobayashi,et al.  Chaotic Rotor Associative Memory , 2009 .

[8]  Igor N. Aizenberg.  A Multi-valued Neuron with a Periodic Activation Function , 2009, IJCCI.

[9]  Igor N. Aizenberg.  Periodic Activation Function and a Modified Learning Algorithm for the Multivalued Neuron , 2010, IEEE Transactions on Neural Networks.

[10]  Richard S. Zemel,et al.  Lending direction to neural networks , 1995, Neural Networks.

[11]  J. Nagumo,et al.  On a response characteristic of a mathematical neuron model , 1972, Kybernetik.

[12]  Masaki Kobayashi,et al.  Gradient Descent Learning for Rotor Associative Memory , 2011 .

[13]  Yukio Kosugi,et al.  An image storage system using complex-valued associative memories , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[14]  Joos Vandewalle,et al.  Multi-Valued and Universal Binary Neurons , 2000 .

[15]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[16]  Noest.  Discrete-state phasor neural networks. , 1988, Physical review. A, General physics.

[17]  Jacek M. Zurada,et al.  A new design method for the complex-valued multistate Hopfield associative memory , 2003, IEEE Trans. Neural Networks.

[18]  Masaki Kobayashi,et al.  Fundamental Abilities of Rotor Associative Memory , 2010, 2010 IEEE/ACIS 9th International Conference on Computer and Information Science.