论文信息 - Analysis of Nonlinear Activation Functions for Classification Tasks Using Convolutional Neural Networks

Analysis of Nonlinear Activation Functions for Classification Tasks Using Convolutional Neural Networks

We give an overview of several nonlinear activation functions in deep neural networks that are used to solve various complex machine learning applications. We make the comparison on effectiveness of using these functions and conduct empirical analysis on the Cat/Dog dataset with the aim of clarifying which function produce the best results overall. We also make the prediction by taking the number of input images on CNN model by applying different activation functions at the number of hidden layers.

Payal Pahwa | Aman Dureja

[1] Sepp Hochreiter,et al. Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[2] Yoshua Bengio,et al. Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[3] Tianqi Chen,et al. Empirical Evaluation of Rectified Activations in Convolutional Network , 2015, ArXiv.

[4] Payal Pahwa,et al. Image retrieval techniques: a survey , 2017 .

[5] Geoffrey E. Hinton,et al. ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6] Sepp Hochreiter,et al. Self-Normalizing Neural Networks , 2017, NIPS.

[7] Jian Sun,et al. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8] Raman Arora,et al. Understanding Deep Neural Networks with Rectified Linear Units , 2016, Electron. Colloquium Comput. Complex..