Lie Algebra-Valued Hopfield Neural Networks

This paper introduces Lie algebra-valued Hopfield neural networks, for which the states, outputs, weights and thresholds are all from a Lie algebra. This type of networks represents an alternative generalization of the real-valued neural networks besides the complex-, hyperbolic-, quaternion-, and Clifford-valued neural networks that have been intensively studied over the last few years. The dynamics of these networks from the energy function point of view is studied by giving the expression of such a function and proving that it is indeed an energy function for the proposed network.

[1]  Gerald Sommer,et al.  On Clifford neurons and Clifford multi-layer perceptrons , 2008, Neural Networks.

[2]  Akira Hirose,et al.  Complex-Valued Neural Networks (Studies in Computational Intelligence) , 2006 .

[3]  Akira Hirose.  -Dimensional Vector Neuron and Its Application to the -Bit Parity Problem , 2013 .

[4]  Masaki Kobayashi,et al.  Hyperbolic Hopfield Neural Networks , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Giovanni Muscato,et al.  Multilayer Perceptrons to Approximate Quaternion Valued Functions , 1997, Neural Networks.

[6]  Yasuaki Kuroe,et al.  On energy function for complex-valued neural networks and its applications , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[7]  Eduardo Bayro-Corrochano,et al.  Clifford Hopfield Neural Networks , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[8]  Calin-Adrian Popa.  Lie algebra-valued neural networks , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[9]  T. Nitta.  A backpropagation algorithm for neural networks based an 3D vector product , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[10]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[11]  Yasuaki Kuroe,et al.  On Activation Functions for Complex-Valued Neural Networks - Existence of Energy Functions , 2003, ICANN.

[12]  B. Widrow,et al.  The complex LMS algorithm , 1975, Proceedings of the IEEE.

[13]  Luigi Fortuna,et al.  Neural networks for quaternion-valued function approximation , 1994, Proceedings of IEEE International Symposium on Circuits and Systems - ISCAS '94.

[14]  T. Nitta,et al.  Three-Dimensional Vector Valued Neural Network and its Generalization Ability , 2006 .

[15]  Marcos Eduardo Valle.  A Novel Continuous-Valued Quaternionic Hopfield Neural Network , 2014, 2014 Brazilian Conference on Intelligent Systems.

[16]  Gerald Sommer,et al.  A hyperbolic multilayer perceptron , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[17]  D. Sattinger,et al.  Lie Groups and Algebras with Applications to Physics, Geometry and Mechanics , 1986 .

[18]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[19]  T. Nitta.  Generalization ability of the three-dimensional back-propagation network , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[20]  Qiang Xu,et al.  Applications of Lie groups and Lie algebra to computer vision: A brief survey , 2012, 2012 International Conference on Systems and Informatics (ICSAI2012).

[21]  Danilo P. Mandic,et al.  Complex Valued Nonlinear Adaptive Filters , 2009 .

[22]  T. Nitta.  An extension of the back-propagation algorithm to three dimensions by vector product , 1993, Proceedings of 1993 IEEE Conference on Tools with Al (TAI-93).

[23]  Tohru Nitta,et al.  On the decision boundaries of hyperbolic neurons , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[24]  Yasuaki Kuroe,et al.  Models of clifford recurrent neural networks and their dynamics , 2011, The 2011 International Joint Conference on Neural Networks.

[25]  Yasuaki Kuroe,et al.  Models of Hopfield-Type Clifford Neural Networks and Their Energy Functions - Hyperbolic and Dual Valued Networks - , 2011, ICONIP.

[26]  D. Mandic,et al.  Complex Valued Nonlinear Adaptive Filters: Noncircularity, Widely Linear and Neural Models , 2009 .

[27]  Justin Pearson,et al.  Neural networks in the Clifford domain , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[28]  T. Nitta,et al.  A quaternary version of the back-propagation algorithm , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[29]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[30]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[31]  A. T. Kroode,et al.  Lie Algebras, Part 2: Finite and Infinite Dimensional Lie Algebras and Applications in Physics , 1997 .