Firefly Algorithm-based Hyperparameters Setting of DRNN for Weather Prediction

A Deep Recurrent Neural Networks (DRNN) is powerful to be used in sequential datasets. Quite hard tasks in DRNN is setting the optimum hyperparameters. There are known to be three types of general methods for searching the optimum DRNN hyperparameters: manual, grid, and random searches. However, these types of methods are not the right choice when a prior experience is insufficient. This paper addresses both the optimization and automation of hyperparameters to build its structure. They are carried out using a Firefly Algorithm (FA), one of the metaheuristic methods. The hyperparameters to be optimized and automated are batch size, dense, and total units in each layer. There are three things to consider in doing FA-based optimization in this test, such as designing FA, determining the initialization of fixed hyperparameters from the DRNN, and determining the range of DRNN hyperparameter values. Evaluation using the dataset of weather history recorded by the Max Planck Biogeochemical Institute, which contains 15 attributes, shows that the FA-based hyperparameters setting of DRNN gives a much lower prediction error of 0.111 than the manual tuning (0.475). Based on that result, when using FA for the optimization of DRNN hyperparameters in weather prediction, it reduces the error value, so the prediction results using DRNN are more accurate.

[1]  Yuefei Zhu,et al.  A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks , 2017, IEEE Access.

[2]  Suyanto Suyanto,et al.  Discrete Firefly Algorithm for an Examination Timetabling , 2019, 2019 International Seminar on Research of Information Technology and Intelligent Systems (ISRITI).

[3]  Suyanto Suyanto,et al.  Clustering Nodes and Discretizing Movement to Increase the Effectiveness of HEFA for a CVRP , 2020 .

[4]  Suyanto,et al.  Evolutionary Discrete Firefly Algorithm for Travelling Salesman Problem , 2011, ICAIS.

[5]  Naixue Xiong,et al.  Selecting Hyper-Parameters of Gaussian Process Regression Based on Non-Inertial Particle Swarm Optimization in Internet of Things , 2019, IEEE Access.

[6]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[7]  Oscar Castillo,et al.  Optimization of granulation for fuzzy controllers of autonomous mobile robots using the Firefly Algorithm , 2018, Granular Computing.

[8]  Suyanto,et al.  Discrete Firefly Algorithm for Traveling Salesman Problem: A New Movement Scheme , 2013 .

[9]  Wenhu Tang,et al.  Deep Learning for Daily Peak Load Forecasting–A Novel Gated Recurrent Neural Network Combining Dynamic Time Warping , 2019, IEEE Access.

[10]  Atulya K. Nagar,et al.  Advances in Nature-Inspired Computing and Applications , 2019, EAI/Springer Innovations in Communication and Computing.

[11]  Assaf Hoogi,et al.  Adaptive Estimation of Active Contour Parameters Using Convolutional Neural Networks and Texture Analysis , 2017, IEEE Trans. Medical Imaging.

[12]  Dino Ienco,et al.  Deep Recurrent Neural Networks for Winter Vegetation Quality Mapping via Multitemporal SAR Sentinel-1 , 2018, IEEE Geoscience and Remote Sensing Letters.

[13]  Ruba Talal,et al.  Training Recurrent Neural Networks by a Hybrid PSO-Cuckoo Search Algorithm for Problems Optimization , 2017 .

[14]  Jae-Young Pyun,et al.  Deep Recurrent Neural Networks for Human Activity Recognition , 2017, Sensors.

[15]  Udit Sharma,et al.  Extraction of efficient electrical parameters of solar cell using firefly and cuckoo search algorithm , 2016, 2016 7th India International Conference on Power Electronics (IICPE).

[16]  Marios Savvides,et al.  Reformulating Level Sets as Deep Recurrent Neural Network Approach to Semantic Segmentation , 2017, IEEE Transactions on Image Processing.

[17]  Wonjong Rhee,et al.  On the Difficulty of DNN Hyperparameter Optimization Using Learning Curve Prediction , 2018, TENCON 2018 - 2018 IEEE Region 10 Conference.

[18]  Xiao Xiang Zhu,et al.  Deep Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Chen Li,et al.  Spatial Sequential Recurrent Neural Network for Hyperspectral Image Classification , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[20]  Marko Beko,et al.  Designing Convolutional Neural Network Architecture by the Firefly Algorithm , 2019, 2019 International Young Engineers Forum (YEF-ECE).

[21]  Chunfeng Wang,et al.  An Improved Firefly Algorithm With Specific Probability and Its Engineering Application , 2019, IEEE Access.

[22]  Şebnem Bora,et al.  Parameter tuning in modeling and simulations by using swarm intelligence optimization algorithms , 2017, 2017 9th International Conference on Computational Intelligence and Communication Networks (CICN).

[23]  Travis Desell,et al.  Optimizing Long Short-Term Memory Recurrent Neural Networks Using Ant Colony Optimization to Predict Turbine Engine Vibration , 2017, Appl. Soft Comput..

[24]  Xin-She Yang,et al.  Swarm Intelligence and Bio-Inspired Computation , 2013 .

[25]  Warih Maharani,et al.  Isolated Word Recognition Using Ergodic Hidden Markov Models and Genetic Algorithm , 2012 .

[26]  Soniya Lalwani,et al.  Efficient discrete firefly algorithm for Ctrie based caching of multiple sequence alignment on optimally scheduled parallel machines , 2019, CAAI Trans. Intell. Technol..

[27]  José Ranilla,et al.  Hyper-parameter selection in deep neural networks using parallel particle swarm optimization , 2017, GECCO.

[28]  Gopinath Ganapathy,et al.  Efficient Deep Learning Hyperparameter Tuning Using Cloud Infrastructure: Intelligent Distributed Hyperparameter Tuning with Bayesian Optimization in the Cloud , 2019, 2019 IEEE 12th International Conference on Cloud Computing (CLOUD).

[29]  Suyanto Suyanto,et al.  New Reward-Based Movement to Improve Globally-Evolved BCO in Nurse Rostering Problem , 2019, 2019 International Seminar on Research of Information Technology and Intelligent Systems (ISRITI).

[30]  Hideharu Amano,et al.  Acceleration of Deep Recurrent Neural Networks with an FPGA cluster , 2019, HEART 2019.

[31]  G. K. Jati,et al.  Discrete cuckoo search for traveling salesman problem , 2012, 2012 7th International Conference on Computing and Convergence Technology (ICCCT).

[32]  Suyanto.  An Informed Genetic Algorithm for University Course and Student Timetabling Problems , 2010, ICAISC.

[33]  W. Marsden.  I and J , 2012 .