N E ] 1 0 M ar 2 01 9 Automatically Evolving CNN Architectures Based on Blocks Yanan Sun and

The performance of Convolutional Neural Networks (CNNs) highly relies on their architectures. In order to design a CNN with promising performance, extended expertise in both CNNs and the investigated problem is required, which is not necessarily held by every user interested in CNNs or the problem domain. In this paper, we propose to automatically evolve CNN architectures by using a genetic algorithm based on ResNet blocks and DenseNet blocks. The proposed algorithm is completely automatic in designing CNN architectures, particularly, neither pre-processing before it starts nor post-processing on the designed CNN is needed. Furthermore, the proposed algorithm does not require users with domain knowledge on CNNs, the investigated problem or even genetic algorithms. The proposed algorithm is evaluated on CIFAR10 and CIFAR100 against 18 state-of-the-art peer competitors. Experimental results show that it outperforms state-of-the-art CNNs hand-crafted and CNNs designed by automatic peer competitors in terms of the classification accuracy, and achieves the competitive classification accuracy against semi-automatic peer competitors. In addition, the proposed algorithm consumes much less time than most peer competitors in finding the best CNN architectures.

[1]  Masanori Suganuma,et al.  A genetic programming approach to designing convolutional neural network architectures , 2017, GECCO.

[2]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[4]  P. Alam ‘G’ , 2021, Composites Engineering: An A–Z Guide.

[5]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[6]  Quoc V. Le,et al.  Large-Scale Evolution of Image Classifiers , 2017, ICML.

[7]  Oriol Vinyals,et al.  Hierarchical Representations for Efficient Architecture Search , 2017, ICLR.

[8]  Yong Yu,et al.  Efficient Architecture Search by Network Transformation , 2017, AAAI.

[9]  P. Alam ‘T’ , 2021, Composites Engineering: An A–Z Guide.

[10]  Ramesh Raskar,et al.  Designing Neural Network Architectures using Reinforcement Learning , 2016, ICLR.

[11]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[12]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[13]  Zhang Yi,et al.  Improved Regularity Model-Based EDA for Many-Objective Optimization , 2018, IEEE Transactions on Evolutionary Computation.

[14]  Zhang Yi,et al.  IGD Indicator-Based Evolutionary Algorithm for Many-Objective Optimization Problems , 2018, IEEE Transactions on Evolutionary Computation.

[15]  Junjie Yan,et al.  Practical Network Blocks Design with Q-Learning , 2017, ArXiv.

[16]  Amos J. Storkey,et al.  Training Deep Convolutional Neural Networks to Play Go , 2015, ICML.

[17]  J. Koenderink Q… , 2014, Les noms officiels des communes de Wallonie, de Bruxelles-Capitale et de la communaute germanophone.

[18]  Zhang Yi,et al.  Reference line-based Estimation of Distribution Algorithm for many-objective optimization , 2017, Knowl. Based Syst..

[19]  P. Alam,et al.  R , 1823, The Herodotus Encyclopedia.

[20]  Xaq Pitkow,et al.  Skip Connections Eliminate Singularities , 2017, ICLR.

[21]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[23]  Alan L. Yuille,et al.  Genetic CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Wolfgang Banzhaf,et al.  Genetic Programming: An Introduction , 1997 .

[25]  P. Alam ‘K’ , 2021, Composites Engineering.

[26]  Yoshua Bengio,et al.  Maxout Networks , 2013, ICML.

[27]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  P. Alam ‘A’ , 2021, Composites Engineering: An A–Z Guide.

[29]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[30]  C. Janis,et al.  THE EVOLUTIONARY STRATEGY OF THE EQUIDAE AND THE ORIGINS OF RUMEN AND CECAL DIGESTION , 1976, Evolution; international journal of organic evolution.

[31]  Danna Zhou,et al.  d. , 1840, Microbial pathogenesis.

[32]  이현주 Q. , 2005 .