Automatically Designing Convolutional Neural Network Architecture with Artificial Flora Algorithm

Convolutional neural network has demonstrated high performance in many real-world problems in recent years. However, the results and accuracy of a CNN that are applied for a specific problem are highly controlled by the architecture and its hyperparameters. The process of finding the right set of hyperparameters for the network’s architecture is a very time-consuming process and requires expertise. To address this problem, we present a powerful method that does automatic hyperparameter search for designing CNN architecture. The hyperparameter optimization is performed by the artificial flora optimization algorithm. The proposed framework has the ability to explore different architectures and optimize the values of hyperparameters for a given task. In this research, the proposed framework is validated on MNIST image classification task and it can be concluded from the experimental results that the suggested search method accomplishes promising achievement in this domain.

[1]  Xiaolin Hu,et al.  Recurrent convolutional neural network for object recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[6]  Yan Wang,et al.  Artificial Flora (AF) Optimization Algorithm , 2018 .

[7]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Milan Tuba,et al.  Bare Bones Fireworks Algorithm for Capacitated p-Median Problem , 2018, ICSI.

[9]  Milan Tuba,et al.  Mobile Robot Path Planning by Improved Brain Storm Optimization Algorithm , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[10]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Milan Tuba,et al.  Unmanned Combat Aerial Vehicle Path Planning by Brain Storm Optimization Algorithm , 2018 .

[12]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[13]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Milan Tuba,et al.  Extreme Learning Machines for Data Classification Tuning by Improved Bat Algorithm , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[15]  Zhuowen Tu,et al.  Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree , 2015, AISTATS.

[16]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[17]  Zhuowen Tu,et al.  Deeply-Supervised Nets , 2014, AISTATS.

[18]  Milan Tuba,et al.  Adjusted Fireworks Algorithm Applied to Retinal Image Registration , 2017 .

[19]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[20]  Marko Beko,et al.  Wireless Sensor Network Localization Problem by Hybridized Moth Search Algorithm , 2018, 2018 14th International Wireless Communications & Mobile Computing Conference (IWCMC).

[21]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[22]  Tobias Senst,et al.  Hyper-parameter optimization for convolutional neural network committees based on evolutionary algorithms , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[23]  Mark D. McDonnell,et al.  Enhanced image classification with a fast-learning shallow convolutional neural network , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[24]  Steven R. Young,et al.  Optimizing deep learning hyper-parameters through an evolutionary algorithm , 2015, MLHPC@SC.

[25]  Marko Beko,et al.  Modified Monarch Butterfly Optimization Algorithm for RFID Network Planning , 2018, 2018 6th International Conference on Multimedia Computing and Systems (ICMCS).

[26]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.