A new type of neurons for machine learning

In machine learning, an artificial neural network is the mainstream approach. Such a network consists of many neurons. These neurons are of the same type characterized by the 2 features: (1) an inner product of an input vector and a matching weighting vector of trainable parameters and (2) a nonlinear excitation function. Here, we investigate the possibility of replacing the inner product with a quadratic function of the input vector, thereby upgrading the first-order neuron to the second-order neuron, empowering individual neurons and facilitating the optimization of neural networks. Also, numerical examples are provided to illustrate the feasibility and merits of the second-order neurons. Finally, further topics are discussed.

[1]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[2]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[3]  David A. Medler,et al.  Training redundant artificial neural networks: Imposing biology on technology , 1994, Psychological research.

[4]  Victor Alves,et al.  Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images , 2016, IEEE Transactions on Medical Imaging.

[5]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[6]  Dinggang Shen,et al.  Machine Learning in Medical Imaging , 2012, Lecture Notes in Computer Science.

[7]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[8]  Zachary Chase Lipton.  A Critical Review of Recurrent Neural Networks for Sequence Learning , 2015, ArXiv.

[9]  Wei Zeng,et al.  Shape Classification Using Wasserstein Distance for Brain Morphometry Analysis , 2015, IPMI.

[10]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[11]  Marios Anthimopoulos,et al.  Lung Pattern Classification for Interstitial Lung Diseases Using a Deep Convolutional Neural Network , 2016, IEEE Transactions on Medical Imaging.

[12]  Hugh M Cartwright,et al.  Artificial Neural Networks in Biology and Chemistry - The Evolution of a New Analytical Tool , 2009, Artificial Neural Networks.

[13]  Ricardo A. M. Valentim,et al.  A fuzzy model for processing and monitoring vital signs in ICU patients , 2011, Biomedical engineering online.

[14]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.