Multi-Valued Autoencoders and Classification of Large-Scale Multi-Class Problem

Two-layered neural networks are well known as autoencoders (AEs) in order to reduce the dimensionality of data. AEs are successfully employed as pre-trained layers of neural networks for classification tasks. Most of the existing studies conceived real-valued AEs in real-valued neural networks. This study investigated complex- and quaternion-valued AEs for complex- and quaternion-valued neural networks. Inputs, weights, biases, and outputs in complex-valued AE (CAE) are complex variables, whereas those in quaternion-valued AE (QAE) are quaternions. In both methods, a split-type activation function is used in the hidden and output units. To deal with the images using the proposed methods, pairs of pixels are allotted to complex-valued inputs in the CAE and quartets of pixels are allotted to quaternion-valued inputs in the QAE. Proposed autoencoders are tested and performance compared with conventional AE for several tasks which are encoding/decoding, handwritten numeral recognition and large-scale multi-class classification. Proposed CAE and QAE revealed as good recognition methods for the tasks and outperformed conventional AE with significance performance in case of large-scale multi-class images recognition.

[1]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[2]  Kazuyuki Murase,et al.  Single-layered complex-valued neural network for real-valued classification problems , 2009, Neurocomputing.

[3]  Hiroyuki Aoki.  Applications of Complex-Valued Neural Networks for Image Processing , 2003 .

[4]  Yasuaki Kuroe,et al.  Phase dynamics of complex-valued neural networks and its application to traffic signal control , 2005, Int. J. Neural Syst..

[5]  Kazuyuki Murase,et al.  Real-Time Hand Gesture Recognition Using Complex-Valued Neural Network (CVNN) , 2011, ICONIP.

[6]  Igor N. Aizenberg,et al.  Complex-Valued Neural Networks with Multi-Valued Neurons , 2011, Studies in Computational Intelligence.

[7]  Nicolas Le Bihan,et al.  Polarized Signal Classification by Complex and quaternionic Multi-Layer Perceptrons , 2008, Int. J. Neural Syst..

[8]  Kazuyuki Murase,et al.  Multi-valued autoencoders for multi-valued neural networks , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[9]  Akira Hirose,et al.  Complex-Valued Neural Networks: Theories and Applications , 2003 .

[10]  Amy Loutfi,et al.  Learning Feature Representations with a Cost-Relevant Sparse Autoencoder , 2015, Int. J. Neural Syst..

[11]  Tohru Nitta,et al.  An Extension of the Back-Propagation Algorithm to Complex Numbers , 1997, Neural Networks.

[12]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[13]  Tohru Nitta,et al.  The uniqueness Theorem for Complex-Valued Neural Networks with Threshold Parameters and the Redundancy of the Parameters , 2008, Int. J. Neural Syst..

[14]  Akira Hirose,et al.  Complex-Valued Neural Networks , 2006, Studies in Computational Intelligence.

[15]  Yasuaki Kuroe,et al.  Models of hopfield-type quaternion neural networks and their energy functions , 2005, Int. J. Neural Syst..

[16]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[17]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[18]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[19]  Xiaoming Chen,et al.  A modified error backpropagation algorithm for complex-value neural networks , 2005, Int. J. Neural Syst..

[20]  廣瀬 明,et al.  Complex-valued neural networks : advances and applications , 2013 .

[21]  Tülay Adali,et al.  Approximation by Fully Complex Multilayer Perceptrons , 2003, Neural Computation.

[22]  Jeffrey Pennington,et al.  Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection , 2011, NIPS.

[23]  Naotake Kamiura,et al.  Associative Memory in quaternionic Hopfield Neural Network , 2008, Int. J. Neural Syst..

[24]  Jonas Schreiber.  Complex Valued Neural Networks Utilizing High Dimensional Parameters , 2016 .

[25]  Nobuyuki Matsui,et al.  Quaternionic Multilayer Perceptron with Local Analyticity , 2012, Inf..

[26]  Nobuyuki Matsui,et al.  Quaternion neural network with geometrical operators , 2004, J. Intell. Fuzzy Syst..

[27]  Hiromi Kusamichi,et al.  A New Scheme for Color Night Vision by Quaternion Neural Network , 2004 .

[28]  T. Nitta,et al.  A quaternary version of the back-propagation algorithm , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[29]  Pierre Baldi,et al.  Complex-Valued Autoencoders , 2011, Neural Networks.