Emerging as one of the most contemporary machine learning techniques, deep learning has shown success in areas such as image classification, speech recognition, and even playing games through the use of hierarchical architecture which includes many layers of non-linear information. In this paper, a powerful deep learning pipeline, intelligent deep learning (iDeepLe) is proposed for both regression and classification tasks. iDeepLe is written in Python with the help of various API libraries such as Keras, TensorFlow, and Scikit-Learn. The core idea of the pipeline is inspired by the sequential modeling with considering numerous layers of neurons to build the deep architecture. Each layer in the sequential deep model can perform independently as a module with minimum finitudes and does not limit the performance of the other layers. iDeepLe has the ability of employing grid search, random search, and Bayesian optimization to tune the most significant predictor input variables and hyper-parameters in the deep model via adaptive learning rate optimization algorithms for both accuracy and complexity, while simultaneously solving the unknown parameters of the regression or the classification model. The parallel pipeline of iDeepLe has the capacity to handle big data problems using Apache Spark, Apache Arrow, High Performance Computing (HPC) and GPU-enabled machines as well. In this paper, to show the importance of the optimization in deep learning, an exhaustive study of the impact of hyper-parameters in a simple and a deep model using optimization algorithms with adaptive learning rate was carried out.
[1]
Amir H. Gandomi,et al.
Genetic Programming Based on Error Decomposition: A Big Data Approach
,
2017,
GPTP.
[2]
Boris Polyak.
Some methods of speeding up the convergence of iteration methods
,
1964
.
[3]
Amir H. Gandomi,et al.
An evolutionary approach for fMRI big data classification
,
2017,
2017 IEEE Congress on Evolutionary Computation (CEC).
[4]
Geoffrey E. Hinton,et al.
Deep Learning
,
2015,
Nature.
[5]
Yee Whye Teh,et al.
A Fast Learning Algorithm for Deep Belief Nets
,
2006,
Neural Computation.
[6]
Amir H. Gandomi,et al.
Building energy consumption forecast using multi-objective genetic programming
,
2018
.
[7]
Simon Fong,et al.
Eidetic Wolf Search Algorithm with a global memory structure
,
2016,
Eur. J. Oper. Res..
[8]
Thomas Hanne,et al.
Nonessential objectives within network approaches for MCDM
,
2006,
Eur. J. Oper. Res..
[9]
Carl E. Rasmussen,et al.
Gaussian processes for machine learning
,
2005,
Adaptive computation and machine learning.
[10]
Simon Fong,et al.
Multi-stage optimization of a deep model: A case study on ground motion modeling
,
2018,
PloS one.
[11]
Nicolas Huck,et al.
Deep neural networks, gradient-boosted trees, random forests: Statistical arbitrage on the S&P 500
,
2017,
Eur. J. Oper. Res..