A Generalization Method of Partitioned Activation Function for Complex Number

A method to convert real number partitioned activation function into complex number one is provided. The method has 4em variations; 1 has potential to get holomorphic activation, 2 has potential to conserve complex angle, and the last 1 guarantees interaction between real and imaginary parts. The method has been applied to LReLU and SELU as examples. The complex number activation function is an building block of complex number ANN, which has potential to properly deal with complex number problems. But the complex activation is not well established yet. Therefore, we propose a way to extend the partitioned real activation to complex number.

[1]  Tülay Adali,et al.  Complex backpropagation neural network using elementary transcendental activation functions , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[2]  Cris Koutsougeras,et al.  Complex domain backpropagation , 1992 .

[3]  Andrew L. Maas.  Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[4]  D. Mandic,et al.  Complex Valued Nonlinear Adaptive Filters: Noncircularity, Widely Linear and Neural Models , 2009 .

[5]  Kazuyuki Murase,et al.  Single-layered complex-valued neural network for real-valued classification problems , 2009, Neurocomputing.

[6]  Diana Thomson La Corte,et al.  Newton's method backpropagation for complex-valued holomorphic multilayer perceptrons , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[7]  Stella X. Yu,et al.  Better than real: Complex-valued neural nets for MRI fingerprinting , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[8]  Hamid A. Jalab,et al.  New activation functions for complex-valued neural network , 2011 .

[9]  L. Corte,et al.  Newton's Method Backpropagation for Complex- Valued Holomorphic Neural Networks: Algebraic and Analytic Properties , 2014 .

[10]  Tohru Nitta,et al.  An Extension of the Back-Propagation Algorithm to Complex Numbers , 1997, Neural Networks.

[11]  Prem Kumar Kalra,et al.  High Dimensional Neural Networks and Applications , 2010, Intelligent Autonomous Systems.

[12]  Tülay Adali,et al.  Approximation by Fully Complex Multilayer Perceptrons , 2003, Neural Computation.

[13]  Raffaele Parisi,et al.  Generalized Splitting 2D Flexible Activation Function , 2003, WIRN.