Stimulating Neurons with another Activation Function

Artificial Intelligence, Machine Learning and Deep Learning have become one of the most popular and useful areas in the current era of computer science. Many tasks seemed impossible in the past with the computers are not easily attained with the help of deep learning models. Each deep learning models relies on a set of layers which produces output based on associated activation function. This paper discusses various existing activation function along with new suggested activation function which is producing considerable accuracy for binary classification using CNN with variety of datasets such as EEG signal classification, popular MNIST digit recogniser and flower classification. The activation function proposed in this paper is different than all the existing ones and still manages to perform much better than all the existing activation functions.

[1]  I.S. Isa,et al.  Suitable MLP Network Activation Functions for Breast Cancer and Thyroid Disease Detection , 2010, 2010 Second International Conference on Computational Intelligence, Modelling and Simulation.

[2]  Andres Ramos,et al.  Security Constrained Unit Commitment Using Line Outage Distribution Factors , 2018, IEEE Transactions on Power Systems.

[3]  Yan Du,et al.  Achieving 100x Acceleration for N-1 Contingency Screening With Uncertain Scenarios Using Deep Convolutional Neural Network , 2019, IEEE Transactions on Power Systems.

[4]  Darshak B. Mehta,et al.  Effect of Different Activation Functions on EEG Signal Classification based on Neural Networks , 2020, 2020 Fourth International Conference on Computing Methodologies and Communication (ICCMC).

[5]  Hong He,et al.  Algorithm Research on Improving Activation Function of Convolutional Neural Networks , 2019, 2019 Chinese Control And Decision Conference (CCDC).

[6]  Fangxing Li,et al.  From AlphaGo to Power System AI: What Engineers Can Learn from Solving the Most Complex Board Game , 2018, IEEE Power and Energy Magazine.

[7]  S. Jeyanthi,et al.  Implementation of single neuron using various activation functions with FPGA , 2014, 2014 IEEE International Conference on Advanced Communications, Control and Computing Technologies.

[8]  Konstantinos G. Derpanis,et al.  Evaluation of deep convolutional nets for document image classification and retrieval , 2015, 2015 13th International Conference on Document Analysis and Recognition (ICDAR).