Uniqueness theorem for quaternionic neural networks

Abstract Quaternion algebra is a four-dimensional extension of the complex number field. The construction of artificial neural networks using complex numbers has recently been extended to that using quaternions. However, there have been few theoretical results for quaternionic neural networks. In the present work, we prove the applicability of the uniqueness theorem to quaternionic neural networks. Uniqueness theorems are important theories related to the singularities of neural networks. We provide the quaternionic versions of several important ideas, such as reducibility and equivalence, for proof of the uniqueness theorem. We can determine all the irreducible quaternionic neural networks that are I/O-equivalent to a given irreducible quaternionic neural network due to the uniqueness theorem.

[1]  Héctor J. Sussmann,et al.  Uniqueness of the weights for minimal feedforward nets with a given input-output map , 1992, Neural Networks.

[2]  Tohru Nitta,et al.  The uniqueness theorem for complex-valued neural networks and the redundancy of the parameters , 2003, Systems and Computers in Japan.

[3]  Masaki Kobayashi,et al.  Twisted quaternary neural networks , 2012 .

[4]  T. Nitta.  An Extension of the Back-propagation Algorithm to Quaternions , 1996 .

[5]  Ryohei Nakano,et al.  A yet faster version of complex-valued multilayer perceptron learning using singular regions and search pruning , 2015, 2015 7th International Joint Conference on Computational Intelligence (IJCCI).

[6]  Tohru Nitta,et al.  Redundancy of the parameters of the complex-valued neural network , 2002, Neurocomputing.

[7]  Ryohei Nakano,et al.  Fast and Stable Learning Utilizing Singular Regions of Multilayer Perceptron , 2013, Neural Processing Letters.

[8]  Kenji Fukumizu,et al.  Local minima and plateaus in hierarchical structures of multilayer perceptrons , 2000, Neural Networks.

[9]  Yasuaki Kuroe,et al.  Applications of Clifford’s Geometric Algebra , 2013, Advances in Applied Clifford Algebras.

[10]  Danilo P. Mandic,et al.  Performance Bounds of Quaternion Estimators , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Masaki Kobayashi,et al.  Uniqueness Theorem of Complex-Valued Neural Networks with Polar-Represented Activation Function , 2015, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[12]  Tohru Nitta.  A Theoretical Foundation for the Widely Linear Processing of Quaternion-Valued Data , 2013 .

[13]  Tohru Nitta.  Learning Dynamics of a Single Polar Variable Complex-Valued Neuron , 2015, Neural Computation.

[14]  Tohru Nitta.  Complex-valued Neural Networks: Utilizing High-dimensional Parameters , 2009 .

[15]  Masaki Kobayashi,et al.  Exceptional Reducibility of Complex-Valued Neural Networks , 2010, IEEE Transactions on Neural Networks.

[16]  Tohru Nitta,et al.  Reducibility of the Complex-valued Neural Network , 2004 .

[17]  Masaki Kobayashi,et al.  Hybrid Quaternionic Hopfield Neural Network , 2015, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[18]  Akira Hirose,et al.  Quaternion Neural-Network-Based PolSAR Land Classification in Poincare-Sphere-Parameter Space , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Tohru Nitta,et al.  The uniqueness Theorem for Complex-Valued Neural Networks with Threshold Parameters and the Redundancy of the Parameters , 2008, Int. J. Neural Syst..

[20]  Naotake Kamiura,et al.  On the Scheme of Quaternionic Multistate Hopfield Neural Network , 2008 .

[21]  P. Arena,et al.  Neural Networks in Multidimensional Domains: Fundamentals and New Trends in Modelling and Control , 1998 .

[22]  Masaki Kobayashi,et al.  Construction of high-dimensional neural networks by linear connections of matrices , 2003 .

[23]  Naotake Kamiura,et al.  Associative Memory in quaternionic Hopfield Neural Network , 2008, Int. J. Neural Syst..

[24]  Nobuyuki Matsui,et al.  Quaternion neural network with geometrical operators , 2004, J. Intell. Fuzzy Syst..

[25]  Hiromi Kusamichi,et al.  A New Scheme for Color Night Vision by Quaternion Neural Network , 2004 .

[26]  T. Nitta,et al.  A Solution to the 4-bit Parity Problem with a Single Quaternary Neuron , 2004 .

[27]  Nobuyuki Matsui,et al.  Quaternionic Multilayer Perceptron with Local Analyticity , 2012, Inf..

[28]  P. Arena,et al.  Neural networks in multidimensional domains , 1998 .

[29]  Akira Hirose,et al.  Complex-Valued Neural Networks: Theories and Applications , 2003 .

[30]  Tohru Nitta,et al.  Local minima in hierarchical structures of complex-valued neural networks , 2013, Neural Networks.

[31]  Akira Hirose,et al.  Complex-Valued Neural Networks , 2006, Studies in Computational Intelligence.

[32]  Akira Hirose,et al.  Complex-Valued Neural Networks: Advances and Applications , 2013 .