Parallel computing method of deep belief networks and its application to traffic flow prediction

Abstract Deep belief networks (DBNs) with outstanding advantages of learning input data features have attained particular attention and are applied widely in image processing, speech recognition, natural language interpretation, disease diagnosis, among others. However, owing to large data, the training processes of DBNs are time-consuming and may not satisfy the requirements of real-time application systems. In this study, a single dataset is decomposed into multiple subdatasets that are distributed to multiple computing nodes. Multiple computing nodes learn the features of their own subdatasets. On the precondition of the remaining features where one computing node learns from the total dataset, the single dataset learning models and algorithms are extended to the cases where multiple computing nodes learn multiple subdatasets in a parallel manner. Learning models and algorithms are proposed for the parallel computing of DBN learning processes. A master–slave parallel computing structure is designed, where the slave computing nodes learn the features of their respective subdatasets and transmit them to the master computing node. The master computing node is critical in synthesizing the learned features from the respective slave computing nodes. The broadcast, synchronization, and synthesis are repeated until all features of subdatasets have been learned. The proposed parallel computing method is applied to traffic flow prediction using practical traffic flow data. Our experimental results verify the effectiveness of the parallel computing method of DBN learning processes in terms of decreasing pre-training and fine-tuning times and maintaining the prominent feature learning abilities.

[1]  Bo Wang,et al.  When Ensemble Learning Meets Deep Learning: a New Deep Support Vector Machine for Classification , 2016, Knowl. Based Syst..

[2]  U. Rajendra Acharya,et al.  Automated detection of coronary artery disease using different durations of ECG segments with convolutional neural network , 2017, Knowl. Based Syst..

[3]  Jana-Rebecca Rehse,et al.  Predicting process behaviour using deep learning , 2016, Decis. Support Syst..

[4]  Guoqiang Han,et al.  δ-agree AdaBoost stacked autoencoder for short-term traffic flow forecasting , 2017, Neurocomputing.

[5]  Lior Wolf,et al.  Channel-Level Acceleration of Deep Face Representations , 2015, IEEE Access.

[6]  Pingyu Jiang,et al.  A deep learning approach for relationship extraction from interaction context in social manufacturing paradigm , 2016, Knowl. Based Syst..

[7]  Yan Tian,et al.  LSTM-based traffic flow prediction with missing data , 2018, Neurocomputing.

[8]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[9]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[10]  Young-Jin Kim,et al.  An architecture for emergency event prediction using LSTM recurrent neural networks , 2018, Expert Syst. Appl..

[11]  Giancarlo Fortino,et al.  Facial Expression Recognition Utilizing Local Direction-Based Robust Features and Deep Belief Network , 2017, IEEE Access.

[12]  Feifeng Zheng,et al.  Forecasting urban traffic flow by SVR with continuous ACO , 2011 .

[13]  Gregory Francis,et al.  Using a model of human visual perception to improve deep learning , 2018, Neural Networks.

[14]  Tao Li,et al.  Deep belief echo-state network and its application to time series prediction , 2017, Knowl. Based Syst..

[15]  Wenhao Huang,et al.  Deep Architecture for Traffic Flow Prediction: Deep Belief Networks With Multitask Learning , 2014, IEEE Transactions on Intelligent Transportation Systems.

[16]  Ivor W. Tsang,et al.  Learning word dependencies in text by means of a deep recurrent belief network , 2016, Knowl. Based Syst..

[17]  Xiaohong Chen,et al.  Short Term Prediction of Freeway Exiting Volume Based on SVM and KNN , 2015 .

[18]  Pravin Varaiya,et al.  Guaranteed prediction and estimation of the state of a road network , 2012 .

[19]  Wanqing Li,et al.  Forecasting short-term traffic speed based on multiple attributes of adjacent roads , 2019, Knowl. Based Syst..

[20]  Shan Wu,et al.  A neural generative autoencoder for bilingual word embeddings , 2018, Inf. Sci..

[21]  B. G. Cetiner,et al.  A NEURAL NETWORK BASED TRAFFIC-FLOW PREDICTION MODEL , 2010 .

[22]  Jia Liu,et al.  Maxout neurons for deep convolutional and LSTM neural networks in speech recognition , 2016, Speech Commun..

[23]  Shu Zhan,et al.  Robust face detection using local CNN and SVM based on kernel combination , 2016, Neurocomputing.

[24]  Nicholas G. Polson,et al.  Deep learning for short-term traffic flow prediction , 2016, 1604.04527.

[25]  Junyu Dong,et al.  Dual channel LSTM based multi-feature extraction in gait for diagnosis of Neurodegenerative diseases , 2018, Knowl. Based Syst..

[26]  Bin Ran,et al.  A hybrid deep learning based traffic flow prediction method and its understanding , 2018 .

[27]  Abhigyan Nath,et al.  Enhanced prediction of recombination hotspots using input features extracted by class specific autoencoders. , 2018, Journal of theoretical biology.

[28]  Da Liu,et al.  Emotional image color transfer via deep learning , 2018, Pattern Recognit. Lett..