Evolving Deep CNN-LSTMs for Inventory Time Series Prediction

Inventory forecasting is a key component of effective inventory management. In this work, we utilise hybrid deep learning models for inventory forecasting. According to the highly nonlinear and non-stationary characteristics of inventory data, the models employ Long Short-Term Memory (LSTM) to capture long temporal dependencies and Convolutional Neural Network (CNN) to learn the local trend features. However, designing optimal CNN-LSTM network architecture and tuning parameters can be challenging and would require consistent human supervision. To automate optimal architecture searching of CNN-LSTM, we implement three meta-heuristics: a Particle Swarm Optimisation (PSO) and two Differential Evolution (DE) variants. Computational experiments on real-world inventory forecasting problems are conducted to evaluate the performance of the applied meta-heuristics in terms of evolved network architectures for obtaining prediction accuracy. Moreover, the evolved CNN-LSTM models are also compared to Seasonal Auto-regressive Integrated Moving Average (SARIMA) models for inventory forecasting problems. The experimental results indicate that the evolved CNN-LSTM models are capable of dealing with complex nonlinear inventory forecasting problem.

[1]  Ponnuthurai N. Suganthan,et al.  Recent advances in differential evolution - An updated survey , 2016, Swarm Evol. Comput..

[2]  Mauro Birattari,et al.  The irace Package: Iterated Race for Automatic Algorithm , 2011 .

[3]  Steven Nahmias,et al.  Production and operations analysis , 1992 .

[4]  Pradip Kumar Bala,et al.  Improving inventory performance with clustering based demand forecasts , 2012 .

[5]  Xun Gong,et al.  Traffic flow forecasting based on hybrid deep learning framework , 2017, 2017 12th International Conference on Intelligent Systems and Knowledge Engineering (ISKE).

[6]  Quoc V. Le,et al.  Large-Scale Evolution of Image Classifiers , 2017, ICML.

[7]  Grazziela Patrocinio Figueredo,et al.  A Simulation-based Optimisation Approach for Inventory Management of Highly Perishable Food , 2019, ICORES.

[8]  H. Russell Fogler,et al.  A Pattern Recognition Model for Forecasting , 1974 .

[9]  Trevor Darrell,et al.  Long-term recurrent convolutional networks for visual recognition and description , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Siti Mariyam Shamsuddin,et al.  Particle Swarm Optimization: Technique, System and Challenges , 2011 .

[11]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[12]  Tara N. Sainath,et al.  Convolutional, Long Short-Term Memory, fully connected Deep Neural Networks , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[14]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Silvia Curteanu,et al.  Optimization methodology based on neural networks and self-adaptive differential evolution algorithm applied to an aerobic fermentation process , 2013, Appl. Soft Comput..

[16]  Grazziela Patrocinio Figueredo,et al.  A Genetic Algorithm With Composite Chromosome for Shift Assignment of Part-time Employees , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[17]  Christopher Joseph Pal,et al.  Delving Deeper into Convolutional Networks for Learning Video Representations , 2015, ICLR.

[18]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[19]  R. Storn,et al.  On the usage of differential evolution for function optimization , 1996, Proceedings of North American Fuzzy Information Processing.

[20]  Bin Wang,et al.  Evolving Deep Convolutional Neural Networks by Variable-Length Particle Swarm Optimization for Image Classification , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[21]  Li Deng,et al.  Ensemble deep learning for speech recognition , 2014, INTERSPEECH.

[22]  Wei Xu,et al.  CNN-RNN: A Unified Framework for Multi-label Image Classification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  H. Akaike Likelihood of a model and information criteria , 1981 .

[24]  Xiaoli Li,et al.  Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition , 2015, IJCAI.

[25]  Adel M. Alimi,et al.  Hierarchical multi-dimensional differential evolution for the design of beta basis function neural network , 2012, Neurocomputing.

[26]  Alex S. Fukunaga,et al.  Reevaluating Exponential Crossover in Differential Evolution , 2014, PPSN.

[27]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Marti J. Anderson,et al.  A new method for non-parametric multivariate analysis of variance in ecology , 2001 .

[29]  Sung-Kwun Oh,et al.  Design of optimized cascade fuzzy controller based on differential evolution: Simulation studies and practical insights , 2012, Eng. Appl. Artif. Intell..

[30]  Tao Lin,et al.  Hybrid Neural Networks for Learning the Trend in Time Series , 2017, IJCAI.

[31]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[32]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[33]  Wei Shen,et al.  Real-time road traffic fusion and prediction with GPS and fixed-sensor data , 2012, 2012 15th International Conference on Information Fusion.

[34]  Donya Rahmani,et al.  An aggregate production planning model for two phase production systems: Solving with genetic algorithm and tabu search , 2012, Expert Syst. Appl..

[35]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[36]  Charles Elkan,et al.  Learning to Diagnose with LSTM Recurrent Neural Networks , 2015, ICLR.