Fully Complex-Valued Dendritic Neuron Model

A single dendritic neuron model (DNM) that owns the nonlinear information processing ability of dendrites has been widely used for classification and prediction. Complex-valued neural networks that consist of a number of multiple/deep-layer McCulloch-Pitts neurons have achieved great successes so far since neural computing was utilized for signal processing. Yet no complex value representations appear in single neuron architectures. In this article, we first extend DNM from a real-value domain to a complex-valued one. Performance of complex-valued DNM (CDNM) is evaluated through a complex XOR problem, a non-minimum phase equalization problem, and a real-world wind prediction task. Also, a comparative analysis on a set of elementary transcendental functions as an activation function is implemented and preparatory experiments are carried out for determining hyperparameters. The experimental results indicate that the proposed CDNM significantly outperforms real-valued DNM, complex-valued multi-layer perceptron, and other complex-valued neuron models.

[1]  Mengchu Zhou,et al.  Decision-Tree-Initialized Dendritic Neuron Model for Fast and Accurate Data Classification , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Huanxin Zou,et al.  Superpixel-Oriented Classification of PolSAR Images Using Complex-Valued Convolutional Neural Network Driven by Hybrid Data , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Haitao Yuan,et al.  Profit-Maximized Collaborative Computation Offloading and Resource Allocation in Distributed Cloud and Edge Computing Systems , 2021, IEEE Transactions on Automation Science and Engineering.

[4]  Quanwang Wu,et al.  A Hybrid Probabilistic Multiobjective Evolutionary Algorithm for Commercial Recommendation Systems , 2021, IEEE Transactions on Computational Social Systems.

[5]  Jiujun Cheng,et al.  Chaotic Local Search-Based Differential Evolution Algorithms for Optimization , 2021, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[6]  Yong Xu,et al.  Image denoising using complex-valued deep CNN , 2021, Pattern Recognit..

[7]  G. Lo,et al.  An optical neural chip for implementing complex-valued neural network , 2021, Nature Communications.

[8]  Yang Yu,et al.  A multi-layered gravitational search algorithm for function optimization and real-world problems , 2021, IEEE/CAA Journal of Automatica Sinica.

[9]  Haijun Jiang,et al.  Nonseparation Method-Based Finite/Fixed-Time Synchronization of Fully Complex-Valued Discontinuous Neural Networks , 2020, IEEE Transactions on Cybernetics.

[10]  Zhen Wang,et al.  Exponential Synchronization of Delayed Memristor-Based Uncertain Complex-Valued Neural Networks for Image Protection , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Konrad P. Kording,et al.  Can Single Neurons Solve MNIST? The Computational Power of Biological Dendritic Trees , 2020 .

[12]  Chong Lin,et al.  Lagrange Exponential Stability of Complex-Valued BAM Neural Networks With Time-Varying Delays , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[13]  Gregory M. P. O'Hare,et al.  A photovoltaic power forecasting model based on dendritic neuron networks with the aid of wavelet transform , 2020, Neurocomputing.

[14]  Ya Tu,et al.  Complex-Valued Networks for Automatic Modulation Classification , 2020, IEEE Transactions on Vehicular Technology.

[15]  Shangce Gao,et al.  A novel machine learning technique for computer-aided diagnosis , 2020, Eng. Appl. Artif. Intell..

[16]  Athanasia Papoutsi,et al.  Illuminating dendritic function with computational models , 2020, Nature Reviews Neuroscience.

[17]  Cong Liu,et al.  A Connectivity-Prediction-Based Dynamic Clustering Model for VANET in an Urban Scene , 2020, IEEE Internet of Things Journal.

[18]  Stanislaw Purgal,et al.  Improving Expressivity of Graph Neural Networks , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[19]  Saumik Bhattacharya,et al.  Co-VeGAN: Complex-Valued Generative Adversarial Network for Compressive Sensing MR Image Reconstruction , 2020, ArXiv.

[20]  Xiaoyi Jiang,et al.  Circular Complex-Valued GMDH-Type Neural Network for Real-Valued Classification Problems , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[21]  He Huang,et al.  Adaptive complex-valued stepsize based fast learning of complex-valued neural networks , 2020, Neural Networks.

[22]  Zhigang Zeng,et al.  Memory analysis for memristors and memristive recurrent neural networks , 2020, IEEE/CAA Journal of Automatica Sinica.

[23]  Dong Liang,et al.  DeepcomplexMRI: Exploiting deep residual network for fast parallel MR imaging with complex convolution. , 2020, Magnetic resonance imaging.

[24]  Simone Scardapane,et al.  Complex-Valued Neural Networks With Nonparametric Activation Functions , 2018, IEEE Transactions on Emerging Topics in Computational Intelligence.

[25]  Joni Dambre,et al.  Dendritic Computation in a Point Neuron Model , 2020, ICANN.

[26]  Yuanyuan Zhou,et al.  CSR-Net: A Novel Complex-Valued Network for Fast and Precise 3-D Microwave Sparse Reconstruction , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[27]  Akira Hirose,et al.  Millimeter-Wave Coherent Imaging of Moving Targets by Using Complex-Valued Self-Organizing Map and Auto-Encoder , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[28]  Shangce Gao,et al.  A review of applications of artificial intelligent algorithms in wind farms , 2019, Artificial Intelligence Review.

[29]  Yuki Todo,et al.  Neurons with Multiplicative Interactions of Nonlinear Synapses , 2019, Int. J. Neural Syst..

[30]  MengChu Zhou,et al.  Comprehensive Learning Particle Swarm Optimization Algorithm With Local Search for Multimodal Functions , 2019, IEEE Transactions on Evolutionary Computation.

[31]  Jianbiao He,et al.  Cv-CapsNet: Complex-Valued Capsule Network , 2019, IEEE Access.

[32]  Huan Su,et al.  Stabilization of Stochastic Uncertain Complex-Valued Delayed Networks via Aperiodically Intermittent Nonlinear Control , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[33]  Jiujun Cheng,et al.  Dendritic Neuron Model With Effective Learning Algorithms for Classification, Approximation, and Prediction , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Xiang Li,et al.  Enhanced Radar Imaging Using a Complex-Valued Convolutional Neural Network , 2017, IEEE Geoscience and Remote Sensing Letters.

[35]  Ahmed Tealab,et al.  Time series forecasting using artificial neural networks methodologies: A systematic review , 2018, Future Computing and Informatics Journal.

[36]  Yan Liu,et al.  A Split-Complex Valued Gradient-Based Descent Neuro-Fuzzy Algorithm for TS System and Its Convergence , 2018, Neural Processing Letters.

[37]  Haijun Jiang,et al.  Quasi-projective synchronization of fractional-order complex-valued recurrent neural networks , 2018, Neural Networks.

[38]  Calin-Adrian Popa,et al.  Complex-Valued Deep Belief Networks , 2018, ISNN.

[39]  Angela Yao,et al.  Complex Gated Recurrent Neural Networks , 2018, NeurIPS.

[40]  Liang Qi,et al.  Modified cuckoo search algorithm to solve economic power dispatch optimization problems , 2018, IEEE/CAA Journal of Automatica Sinica.

[41]  Sandeep Subramanian,et al.  Deep Complex Networks , 2017, ICLR.

[42]  Haipeng Wang,et al.  Complex-Valued Convolutional Neural Network and Its Application in Polarimetric SAR Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[43]  Masaki Kobayashi,et al.  Symmetric Complex-Valued Hopfield Neural Networks , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[44]  Ignacio Rojas,et al.  Neural networks: An overview of early research, current frameworks and new challenges , 2016, Neurocomputing.

[45]  Jiahai Wang,et al.  Financial time series prediction using a dendritic neuron model , 2016, Knowl. Based Syst..

[46]  Shaista Hussain,et al.  Multiclass Classification by Adaptive Network of Dendritic Neurons with Binary Synapses Using Structural Plasticity , 2016, Front. Neurosci..

[47]  Alex Graves,et al.  Associative Long Short-Term Memory , 2016, ICML.

[48]  Jiujun Cheng,et al.  An approximate logic neuron model with a dendritic structure , 2016, Neurocomputing.

[49]  Mark Tygert,et al.  A Mathematical Motivation for Complex-Valued Convolutional Networks , 2015, Neural Computation.

[50]  Shaista Hussain,et al.  Hardware-Amenable Structural Learning for Spike-Based Pattern Classification Using a Simple Model of Active Dendrites , 2014, Neural Computation.

[51]  Zheng Tang,et al.  Unsupervised learnable neuron model with nonlinear interaction on dendrites , 2014, Neural Networks.

[52]  Thomas Serre,et al.  Neuronal Synchrony in Complex-Valued Deep Networks , 2013, ICLR.

[53]  Sundaram Suresh,et al.  A complex-valued neuro-fuzzy inference system and its learning mechanism , 2014, Neurocomputing.

[54]  Kazuyuki Murase,et al.  Orthogonal least squares based complex-valued functional link network , 2012, Neural Networks.

[55]  Sundaram Suresh,et al.  A meta-cognitive learning algorithm for a Fully Complex-valued Relaxation Network , 2012, Neural Networks.

[56]  Sundaram Suresh,et al.  Metacognitive Learning in a Fully Complex-Valued Radial Basis Function Neural Network , 2012, Neural Computation.

[57]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[58]  Akira Hirose,et al.  Generalization Characteristics of Complex-Valued Feedforward Neural Networks in Relation to Signal Coherence , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[59]  Wolfgang Maass,et al.  Probabilistic Inference in General Graphical Models through Sampling in Stochastic Networks of Spiking Neurons , 2011, PLoS Comput. Biol..

[60]  Narasimhan Sundararajan,et al.  A Sequential Learning Algorithm for Complex-Valued Self-Regulating Resource Allocation Network-CSRAN , 2011, IEEE Transactions on Neural Networks.

[61]  Sheng Chen,et al.  Modeling of Complex-Valued Wiener Systems Using B-Spline Neural Network , 2011, IEEE Transactions on Neural Networks.

[62]  Qeethara Al-Shayea.  Artificial Neural Networks in Medical Diagnosis , 2011 .

[63]  Y. Dan,et al.  An arithmetic rule for spatial summation of excitatory and inhibitory inputs in pyramidal neurons , 2009, Proceedings of the National Academy of Sciences.

[64]  Paramasivan Saratchandran,et al.  A new learning algorithm with logarithmic performance index for complex-valued neural networks , 2009, Neurocomputing.

[65]  Akira Hirose,et al.  Complex-valued neural networks: The merits and their origins , 2009, 2009 International Joint Conference on Neural Networks.

[66]  Kazuyuki Aihara,et al.  Complex-valued prediction of wind profile using augmented complex statistics , 2009 .

[67]  Gouhei Tanaka,et al.  Complex-valued multistate associative memory with nonlinear multilevel functions for gray-level image reconstruction , 2009, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[68]  Sammy Siu,et al.  Sensitivity Analysis of the Split-Complex Valued Multilayer Perceptron Due to the Errors of the i.i.d. Inputs and Weights , 2007, IEEE Transactions on Neural Networks.

[69]  Tim Gollisch,et al.  Modeling Single-Neuron Dynamics and Computations: A Balance of Detail and Abstraction , 2006, Science.

[70]  Kazuyuki Aihara,et al.  Complex-valued forecasting of wind profile , 2006 .

[71]  Parham Aarabi,et al.  On the importance of phase in human speech recognition , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[72]  Xiaoming Chen,et al.  A modified error backpropagation algorithm for complex-value neural networks , 2005, Int. J. Neural Syst..

[73]  M. London,et al.  Dendritic computation. , 2005, Annual review of neuroscience.

[74]  E. Marder,et al.  Plasticity in single neuron and circuit computations , 2004, Nature.

[75]  Tohru Nitta,et al.  Orthogonality of Decision Boundaries in Complex-Valued Neural Networks , 2004, Neural Computation.

[76]  Tülay Adali,et al.  Approximation by Fully Complex Multilayer Perceptrons , 2003, Neural Computation.

[77]  Bartlett W. Mel,et al.  Pyramidal Neuron as Two-Layer Neural Network , 2003, Neuron.

[78]  C. Koch,et al.  Multiplicative computation in a visual neuron sensitive to looming , 2002, Nature.

[79]  P. Detwiler,et al.  Directionally selective calcium signals in dendrites of starburst amacrine cells , 2002, Nature.

[80]  Tülay Adali,et al.  Fully Complex Multi-Layer Perceptron Network for Nonlinear Signal Processing , 2002, J. VLSI Signal Process..

[81]  Narasimhan Sundararajan,et al.  Complex-Valued Minimal Resource Allocation Network for Nonlinear Signal Processing , 2000, Int. J. Neural Syst..

[82]  J. Rinzel,et al.  The role of dendrites in auditory coincidence detection , 1998, Nature.

[83]  Tohru Nitta,et al.  An Extension of the Back-Propagation Algorithm to Complex Numbers , 1997, Neural Networks.

[84]  B. S. Lim,et al.  Optimal design of neural networks using the Taguchi method , 1995, Neurocomputing.

[85]  Saleem A. Kassam,et al.  Channel Equalization Using Adaptive Complex Radial Basis Function Networks , 1995, IEEE J. Sel. Areas Commun..

[86]  Cris Koutsougeras,et al.  Complex domain backpropagation , 1992 .

[87]  Francesco Piazza,et al.  On the complex backpropagation algorithm , 1992, IEEE Trans. Signal Process..

[88]  Thomas L. Clarke,et al.  Generalization of neural networks to the complex plane , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[89]  A.V. Oppenheim,et al.  The importance of phase in signals , 1980, Proceedings of the IEEE.