On Efficient Learning Machine With Root-Power Mean Neuron in Complex Domain

This paper describes an artificial neuron structure and an efficient learning procedure in the complex domain. This artificial neuron aims at incorporating an improved aggregation operation on the complex-valued signals. The aggregation operation is based on the idea underlying the weighted root power mean of input signals. This aggregation operation allows modeling the degree of compensation in a natural manner and includes various aggregation operations as its special cases. The complex resilient propagation algorithm (C-RPROP) with error-dependent weight backtracking step accelerates the training speed significantly and provides better approximation accuracy. Finally, performance evaluation of the proposed complex root power mean neuron with the C-RPROP learning algorithm on various typical examples is given to understand the motivation.

[1]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[2]  D. Mandic,et al.  Complex Valued Nonlinear Adaptive Filters: Noncircularity, Widely Linear and Neural Models , 2009 .

[3]  Tohru Nitta.  An Analysis of the Fundamental Structure of Complex-Valued Neurons , 2004, Neural Processing Letters.

[4]  Madan M. Gupta,et al.  Static and Dynamic Neural Networks: From Fundamentals to Advanced Theory , 2003 .

[5]  David B. Fogel.  An information criterion for optimal neural network selection , 1991, IEEE Trans. Neural Networks.

[6]  Devendra K. Chaturvedi,et al.  Improved generalized neuron model for short-term load forecasting , 2003, Soft Comput..

[7]  George M. Georgiou,et al.  Exact Interpolation and Learning in Quadratic Neural Networks , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[8]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[9]  Danilo P. Mandic,et al.  Complex Valued Nonlinear Adaptive Filters , 2009 .

[10]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[11]  Francesco Piazza,et al.  On the complex backpropagation algorithm , 1992, IEEE Trans. Signal Process..

[12]  Ken Kreutz-Delgado,et al.  The Complex Gradient Operator and the CR-Calculus ECE275A - Lecture Supplement - Fall 2005 , 2009, 0906.4835.

[13]  Ming Zhang,et al.  Neuron-adaptive higher order neural-network models for automated financial data modeling , 2002, IEEE Trans. Neural Networks.

[14]  Christian Igel,et al.  Empirical evaluation of the improved Rprop learning algorithms , 2003, Neurocomputing.

[15]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[16]  Akira Hirose,et al.  Complex-Valued Neural Networks , 2006, Studies in Computational Intelligence.

[17]  T. Poggio,et al.  Multiplying with synapses and neurons , 1992 .

[18]  W. Pedrycz,et al.  Generalized means as model of compensative connectives , 1984 .

[19]  J.G. Daugman,et al.  Entropy reduction and decorrelation in visual coding by oriented neural receptive fields , 1989, IEEE Transactions on Biomedical Engineering.

[20]  Chien-Kuo Li.  A Sigma-Pi-Sigma Neural Network (SPSNN) , 2004, Neural Processing Letters.

[21]  Prem Kumar Kalra,et al.  Some new neural network architectures with improved learning schemes , 2000, Soft Comput..

[22]  George D. Magoulas,et al.  Improving the Convergence of the Backpropagation Algorithm Using Learning Rate Adaptation Methods , 1999, Neural Computation.

[23]  Michael T. Manry,et al.  A Functional Link Network With Ordered Basis Functions , 2007, 2007 International Joint Conference on Neural Networks.

[24]  Tohru Nitta,et al.  An Extension of the Back-Propagation Algorithm to Complex Numbers , 1997, Neural Networks.

[25]  Tülay Adali,et al.  Approximation by Fully Complex Multilayer Perceptrons , 2003, Neural Computation.

[26]  Prem Kumar Kalra,et al.  The Generalized Product Neuron Model in Complex Domain , 2008, ICONIP.

[27]  Prem Kumar Kalra,et al.  New Neuron Model for Blind Source Separation , 2009, ICONIP.

[28]  Xiaoming Chen,et al.  An Modified Error Function for the Complex-value Backpropagation Neural Networks , 2005 .

[29]  Gouhei Tanaka,et al.  Complex-valued multistate associative memory with nonlinear multilevel functions for gray-level image reconstruction , 2009, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[30]  Ganapati Panda,et al.  Nonlinear channel equalization for QAM signal constellation using artificial neural networks , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[31]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[32]  Bartlett W. Mel,et al.  Information Processing in Dendritic Trees , 1994, Neural Computation.

[33]  Prem Kumar Kalra,et al.  The novel aggregation function-based neuron models in complex domain , 2010, Soft Comput..

[34]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[35]  Madan M. Gupta,et al.  Study on general second-order neural units (SONUs) , 2002, Proceedings of the 5th Biannual World Automation Congress.

[36]  Sammy Siu,et al.  Analysis of the Initial Values in Split-Complex Backpropagation Algorithm , 2008, IEEE Transactions on Neural Networks.

[37]  Yoan Shin,et al.  A complex pi-sigma network and its application to equalizaton of nonlinear satellite channels , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[38]  Michael Schmitt,et al.  On the Complexity of Computing and Learning with Multiplicative Neural Networks , 2002, Neural Computation.

[39]  Colin Giles,et al.  Learning, invariance, and generalization in high-order neural networks. , 1987, Applied optics.

[40]  Jacek M. Zurada,et al.  Blur Identification by Multilayer Neural Network Based on Multivalued Neurons , 2008, IEEE Transactions on Neural Networks.

[41]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[42]  Brian D. Ripley,et al.  Neural Networks and Related Methods for Classification , 1994 .

[43]  Joydeep Ghosh,et al.  Ridge polynomial networks , 1995, IEEE Trans. Neural Networks.

[44]  Marco Russo,et al.  Genetic fuzzy learning , 2000, IEEE Trans. Evol. Comput..

[45]  J. W. Brown,et al.  Complex Variables and Applications , 1985 .

[46]  Cris Koutsougeras,et al.  Complex domain backpropagation , 1992 .

[47]  Arjen van Ooyen,et al.  Improving the convergence of the back-propagation algorithm , 1992, Neural Networks.

[48]  Scott E. Fahlman,et al.  An empirical study of learning speed in back-propagation networks , 1988 .

[49]  Vivien A. Casagrande,et al.  Biophysics of Computation: Information Processing in Single Neurons , 1999 .

[50]  Mitio Nagumo.  Über eine Klasse der Mittelwerte , 1930 .