High-dimensional information processing through resilient propagation in quaternionic domain

Abstract This paper proposes a fast but effective machine learning technique for the high-dimensional information processing problems. A quaternion is the hyper-complex number in four dimensions which possesses four components in a single body with embedded phase information among their components. Thus, a neural network with quaternion as unit of information flow has ability to learn and generalize magnitude and argument of high-dimensional information simultaneously. The slow convergence and getting stuck into bad minima are the main drawbacks of the back-propagation learning algorithm; therefore learning technique with faster convergence is the demand of quaternionic neurocomputing. The basic issues of quaternionic back-propagation (ℍ-BP) algorithm have been combated by proposed resilient propagation algorithm in quaternionnic domain (ℍ-RPROP). The efficient and intelligent behavior of the proposed learning algorithm has been vindicated by the wide spectrum of benchmark problems. The simulations on different transformations demonstrate its ability to learn 3D motion, which is very desirable in viewing of objects through different orientations in many engineering applications. The critical thing is the training through mapping over straight line having a small number of points and eventually generalization over complicated geometrical structures (Sphere, Cylinder and Torus) possessing huge amount of point cloud data. The 3D and 4D time series prediction as benchmark applications are also taken into the account for the significant contribution of the proposed work. The substantial edge of the proposed algorithm is its quicker convergence with better approximation capability in high-dimensional problems.

[1]  Prem Kumar Kalra,et al.  On Efficient Learning Machine With Root-Power Mean Neuron in Complex Domain , 2011, IEEE Transactions on Neural Networks.

[2]  Li D. Xu,et al.  Enterprise Integration and Information Architecture: A Systems Perspective on Industrial Information Integration , 2014 .

[3]  Prem Kumar Kalra,et al.  Functional mapping with complex higher order compensatory neuron model , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[4]  Calin-Adrian Popa,et al.  Scaled Conjugate Gradient Learning for Quaternion-Valued Neural Networks , 2016, ICONIP.

[5]  Nobuyuki Matsui,et al.  Feed forward neural network with random quaternionic neurons , 2017, Signal Process..

[6]  Jukka Saarinen,et al.  Complex RPROP-algorithm for neural network equalization of GSM data bursts , 2004, Neurocomputing.

[7]  Kazuhiko Takahashi,et al.  Design of control systems using quaternion neural network and its application to inverse kinematics of robot manipulator , 2013, Proceedings of the 2013 IEEE/SICE International Symposium on System Integration.

[8]  Jinde Cao,et al.  Global dissipativity analysis for delayed quaternion-valued neural networks , 2017, Neural Networks.

[9]  Prem Kumar Kalra,et al.  The novel aggregation function-based neuron models in complex domain , 2010, Soft Comput..

[10]  Nobuyuki Matsui,et al.  Quaternion neural network with geometrical operators , 2004, J. Intell. Fuzzy Syst..

[11]  P. Arena,et al.  Quaternionic Multilayer Perceptrons for Chaotic Time Series Prediction , 1996 .

[12]  Yong Chen,et al.  Industrial information integration - A literature review 2006-2015 , 2016, J. Ind. Inf. Integr..

[13]  Akira Hirose,et al.  Complex-Valued Neural Networks , 2006, Studies in Computational Intelligence.

[14]  Leon O. Chua,et al.  The double scroll , 1985 .

[15]  Danilo P. Mandic,et al.  A Quaternion Gradient Operator and Its Applications , 2011, IEEE Signal Processing Letters.

[16]  Danilo P. Mandic,et al.  Quaternion-Valued Nonlinear Adaptive Filtering , 2011, IEEE Transactions on Neural Networks.

[17]  Giovanni Muscato,et al.  Multilayer Perceptrons to Approximate Quaternion Valued Functions , 1997, Neural Networks.

[18]  David B. Fogel An information criterion for optimal neural network selection , 1991, IEEE Trans. Neural Networks.

[19]  E. Lorenz Deterministic nonperiodic flow , 1963 .

[20]  William Rowan Hamilton,et al.  Lectures on quaternions , 1853 .

[21]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[22]  Boonserm Kijsirikul,et al.  Approximate Match of Rules Using Backpropagation Neural Networks , 2001, Machine Learning.

[23]  Christian Igel,et al.  Empirical evaluation of the improved Rprop learning algorithms , 2003, Neurocomputing.

[24]  Klaus Mueller,et al.  The Subspace Voyager: Exploring High-Dimensional Data along a Continuum of Salient 3D Subspaces , 2016, IEEE Transactions on Visualization and Computer Graphics.