Deep Belief Network for Meteorological Time Series Prediction in the Internet of Things

Seeking to address the challenges associated with high-dimensional complex time series representations of recurrent neural networks, such as low generalization ability and long training time, a hybrid neural network based on a deep belief network (DBN) is proposed in this paper to facilitate time series predictions for the Internet of Things. In our approach, we integrate both a DBN and a recurrent neural network with the gated recurrent unit as the activation unit. First, we implement unsupervised pretraining through the DBN and then supervise the curve fitting using the recurrent neural network. Finally, the hybrid neural network is learned and can make predictions. The experimental results show that the hybrid neural network has a stronger historical learning ability than two other widely used recurrent neural networks and can effectively reduce the number of iterations required by the recurrent neural network, thereby reducing the overall learning time.

[1]  Dong Yu,et al.  Deep Learning: Methods and Applications , 2014, Found. Trends Signal Process..

[2]  Pinyi Ren,et al.  Security Enhancement for IoT Communications Exposed to Eavesdroppers With Uncertain Locations , 2016, IEEE Access.

[3]  Houshang Darabi,et al.  LSTM Fully Convolutional Networks for Time Series Classification , 2017, IEEE Access.

[4]  Chen Wang,et al.  CANS: Towards Congestion-Adaptive and Small Stretch Emergency Navigation with Wireless Sensor Networks , 2016, IEEE Transactions on Mobile Computing.

[5]  Shaohua Wan,et al.  Energy-Efficient Adaptive Routing and Context-Aware Lifetime Maximization in Wireless Sensor Networks , 2014, Int. J. Distributed Sens. Networks.

[6]  Kim-Kwang Raymond Choo,et al.  Multi-dimensional data indexing and range query processing via Voronoi diagram for internet of things , 2019, Future Gener. Comput. Syst..

[7]  Lei Chen,et al.  RFID-enabled indoor positioning method for a real-time manufacturing execution system using OS-ELM , 2016, Neurocomputing.

[8]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[9]  Chengjian Zhang,et al.  A note on compact finite difference method for reaction–diffusion equations with delay☆ , 2015 .

[10]  Tapani Raiko,et al.  Semi-supervised Learning with Ladder Networks , 2015, NIPS.

[11]  Yudong Zhang,et al.  On the Construction of Data Aggregation Tree With Maximizing Lifetime in Large-Scale Wireless Sensor Networks , 2016, IEEE Sensors Journal.

[12]  Hwee Pink Tan,et al.  Mobile big data analytics using deep learning and apache spark , 2016, IEEE Network.

[13]  Ju Ren,et al.  Delay-Optimal Proactive Service Framework for Block-Stream as a Service , 2018, IEEE Wireless Communications Letters.

[14]  Liming Chen,et al.  An Open Internet of Things System Architecture Based on Software-Defined Device , 2019, IEEE Internet of Things Journal.

[15]  Mahmoud Daneshmand,et al.  General Cyberspace: Cyberspace and Cyber-Enabled Spaces , 2018, IEEE Internet of Things Journal.

[16]  Mohsen Guizani,et al.  Semisupervised Deep Reinforcement Learning in Support of IoT and Smart City Services , 2018, IEEE Internet of Things Journal.

[17]  Bo Wang,et al.  Power System Transient Stability Assessment Based on Big Data and the Core Vector Machine , 2016, IEEE Transactions on Smart Grid.

[18]  Wei Fan,et al.  Mining big data: current status, and forecast to the future , 2013, SKDD.

[19]  Mehdi Khashei,et al.  A novel hybridization of artificial neural networks and ARIMA models for time series forecasting , 2011, Appl. Soft Comput..

[20]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[21]  Dongfang Li,et al.  A two-level linearized compact ADI scheme for two-dimensional nonlinear reaction-diffusion equations , 2018, Comput. Math. Appl..

[22]  Yin Zhang,et al.  Coverage Hole Bypassing in Wireless Sensor Networks , 2016, MSN.

[23]  Hao Jiang,et al.  An online sequential extreme learning machine approach to WiFi based indoor positioning , 2014, 2014 IEEE World Forum on Internet of Things (WF-IoT).

[24]  Razvan Pascanu,et al.  How to Construct Deep Recurrent Neural Networks , 2013, ICLR.

[25]  C. L. Philip Chen,et al.  A Data-Emergency-Aware Scheduling Scheme for Internet of Things in Smart Cities , 2018, IEEE Transactions on Industrial Informatics.

[26]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[27]  Zhigang Chen,et al.  Utility-Optimal Resource Management and Allocation Algorithm for Energy Harvesting Cognitive Radio Sensor Networks , 2016, IEEE Journal on Selected Areas in Communications.

[28]  Tao Zhu,et al.  Learning enhanced differential evolution for tracking optimal decisions in dynamic power systems , 2017, Appl. Soft Comput..

[29]  Piet Demeester,et al.  Distributed Neural Networks for Internet of Things: The Big-Little Approach , 2015, IoT 360.

[30]  Kitsuchart Pasupa,et al.  Recurrent Kernel Extreme Reservoir Machine for Time Series Prediction , 2018, IEEE Access.

[31]  Ponnuthurai Nagaratnam Suganthan,et al.  Empirical Mode Decomposition based ensemble deep learning for load demand time series forecasting , 2017, Appl. Soft Comput..

[32]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Mianxiong Dong,et al.  Multiobjective Optimization in Cloud Brokering Systems for Connected Internet of Things , 2017, IEEE Internet of Things Journal.

[34]  Ju Ren,et al.  Two Time-Scale Resource Management for Green Internet of Things Networks , 2019, IEEE Internet of Things Journal.

[35]  Mehdi Khashei,et al.  An artificial neural network (p, d, q) model for timeseries forecasting , 2010, Expert Syst. Appl..

[36]  Tao Zhang,et al.  Wearable Depth Camera: Monocular Depth Estimation via Sparse Optimization Under Weak Supervision , 2018, IEEE Access.

[37]  Jianzhou Wang,et al.  Research and application of local perceptron neural network in highway rectifier for time series forecasting , 2018, Appl. Soft Comput..

[38]  Keqiu Li,et al.  How Can Heterogeneous Internet of Things Build Our Future: A Survey , 2018, IEEE Communications Surveys & Tutorials.

[39]  Wai Ho Mow,et al.  An Efficient Algorithm for Optimally Solving a Shortest Vector Problem in Compute-and-Forward Design , 2014, IEEE Transactions on Wireless Communications.

[40]  Mianxiong Dong,et al.  Learning IoT in Edge: Deep Learning for the Internet of Things with Edge Computing , 2018, IEEE Network.

[41]  Peng Zhang,et al.  Energy-Efficient Multi-Constraint Routing Algorithm With Load Balancing for Smart City Applications , 2016, IEEE Internet of Things Journal.

[42]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[43]  Mehdi Khashei,et al.  A new class of hybrid models for time series forecasting , 2012, Expert Syst. Appl..

[44]  Cees T. A. M. de Laat,et al.  Addressing big data issues in Scientific Data Infrastructure , 2013, 2013 International Conference on Collaboration Technologies and Systems (CTS).

[45]  Mohsen Guizani,et al.  Deep Learning for IoT Big Data and Streaming Analytics: A Survey , 2017, IEEE Communications Surveys & Tutorials.

[46]  Kalyani Mali,et al.  A novel data partitioning and rule selection technique for modeling high-order fuzzy time series , 2018, Appl. Soft Comput..

[47]  Daniel F. Leite,et al.  Ensemble of evolving data clouds and fuzzy models for weather time series prediction , 2018, Appl. Soft Comput..

[48]  A. Lapedes,et al.  Nonlinear signal processing using neural networks: Prediction and system modelling , 1987 .

[49]  Zhigang Chen,et al.  Energy-Harvesting-Aided Spectrum Sensing and Data Transmission in Heterogeneous Cognitive Radio Sensor Network , 2016, IEEE Transactions on Vehicular Technology.

[50]  Zhimin Zhang,et al.  Unconditionally Optimal Error Estimates of a Linearized Galerkin Method for Nonlinear Time Fractional Reaction–Subdiffusion Equations , 2018, Journal of Scientific Computing.

[51]  Huajun Chen,et al.  Semantic Framework of Internet of Things for Smart Cities: Case Studies , 2016, Sensors.

[52]  Ajith Abraham,et al.  Weather Forecasting Models Using Ensembles of Neural Networks , 2003 .

[53]  Xiaobo Zhang,et al.  A novel decomposition‐ensemble model for forecasting short‐term load‐time series with multiple seasonal patterns , 2018, Appl. Soft Comput..