Hyperbolic Hopfield Neural Networks

In recent years, several neural networks using Clifford algebra have been studied. Clifford algebra is also called geometric algebra. Complex-valued Hopfield neural networks (CHNNs) are the most popular neural networks using Clifford algebra. The aim of this brief is to construct hyperbolic HNNs (HHNNs) as an analog of CHNNs. Hyperbolic algebra is a Clifford algebra based on Lorentzian geometry. In this brief, a hyperbolic neuron is defined in a manner analogous to a phasor neuron, which is a typical complex-valued neuron model. HHNNs share common concepts with CHNNs, such as the angle and energy. However, HHNNs and CHNNs are different in several aspects. The states of hyperbolic neurons do not form a circle, and, therefore, the start and end states are not identical. In the quantized version, unlike complex-valued neurons, hyperbolic neurons have an infinite number of states.

[1]  André J. Noest,et al.  Phasor Neural Networks , 1987, NIPS.

[2]  Sven Buchholz,et al.  A theory of neural computation with Clifford algebras , 2006 .

[3]  P. Arena,et al.  Neural networks in multidimensional domains , 1998 .

[4]  Yasuaki Kuroe,et al.  Models of Hopfield-Type Clifford Neural Networks and Their Energy Functions - Hyperbolic and Dual Valued Networks - , 2011, ICONIP.

[5]  Masaki Kobayashi,et al.  Construction of high-dimensional neural networks by linear connections of matrices , 2003 .

[6]  Naotake Kamiura,et al.  Associative Memory in quaternionic Hopfield Neural Network , 2008, Int. J. Neural Syst..

[7]  Jacek M. Zurada,et al.  A new design method for the complex-valued multistate Hopfield associative memory , 2003, IEEE Trans. Neural Networks.

[8]  Masaki Kobayashi,et al.  Noise Robust Gradient Descent Learning for Complex-Valued Associative Memory , 2011, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[9]  Richard S. Zemel,et al.  Lending direction to neural networks , 1995, Neural Networks.

[10]  Francesco Piazza,et al.  On the complex backpropagation algorithm , 1992, IEEE Trans. Signal Process..

[11]  Tohru Nitta,et al.  On the decision boundaries of hyperbolic neurons , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[12]  Joos Vandewalle,et al.  Multi-Valued and Universal Binary Neurons: Theory, Learning and Applications , 2012 .

[13]  Tohru Nitta,et al.  An Extension of the Back-Propagation Algorithm to Complex Numbers , 1997, Neural Networks.

[14]  Nobuyuki Matsui,et al.  Commutative quaternion and multistate Hopfield neural networks , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[15]  Gerald Sommer,et al.  A hyperbolic multilayer perceptron , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[16]  T. Nitta.  An Extension of the Back-propagation Algorithm to Quaternions , 1996 .

[17]  H. Ritter.  Self-Organizing Maps on non-euclidean Spaces , 1999 .

[18]  Nobuyuki Matsui,et al.  An iterative learning scheme for multistate complex-valued and quaternionic Hopfield neural networks , 2009, 2009 International Joint Conference on Neural Networks.

[19]  Joos Vandewalle,et al.  Multi-Valued and Universal Binary Neurons , 2000 .

[20]  Donq-Liang Lee.  Improving the capacity of complex-valued neural networks with a modified gradient descent learning rule , 2001, IEEE Trans. Neural Networks.

[21]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Jacek M. Zurada,et al.  Complex-valued multistate neural associative memory , 1996, IEEE Trans. Neural Networks.

[23]  H. Aoki.  A complex-valued neuron to transform gray level images to phase information , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[24]  Akira Hirose,et al.  Continuous complex-valued back-propagation learning , 1992 .

[25]  Helge J. Ritter,et al.  Hyperbolic Self-Organizing Maps for Semantic Navigation , 2001, NIPS.

[26]  Noest.  Discrete-state phasor neural networks. , 1988, Physical review. A, General physics.

[27]  Masaki Kobayashi,et al.  Projection rule for complex-valued associative memory with large constant terms , 2012 .

[28]  Helge J. Ritter,et al.  Text Categorization and Semantic Browsing with Self-Organizing Maps on Non-euclidean Spaces , 2001, PKDD.