Hyperparameter Optimization in a Convolutional Neural Network Using Metaheuristic Algorithms

The hyperparameters of a convolutional neural network always have been important, because the performance of the convolutional neural network depends largely on them. To find manually the optimal value requires a lot of work, experience, and time. Metaheuristic algorithms are approximate general-purpose search and optimization algorithms, they are used to find a good solution into search space. In the book chapter, the hyperparameters of a convolutional neural network are optimized. A metaheuristic algorithm is used to finds the best solution, that is, the best hyperparameter value for higher accuracy of the convolutional neural network (CNN). The results of four metaheuristic algorithms that have the same objective are compared, which are PSO (Particle Swarm Optimization), ABC (Artificial Bee Colony), ALO (Ant Lion Optimization), and BA (Bat Algorithm). Each algorithm uses the same convolutional neural network architecture as well as the same dataset (MNIST database) also each algorithm is run 30 times to perform the corresponding statistics. Finally, the results of the algorithms are shown.