Exploiting the Spatio-Temporal Patterns in IoT Data to Establish a Dynamic Ensemble of Distributed Learners

Internet of Things applications can greatly benefit from accurate prediction models. The performance of prediction models is highly dependent on the quantity and quality of their training data. In this paper, we investigate the creation of a dynamic ensemble from distributed deep learning models by considering the spatiotemporal patterns embedded in the training data. Our dynamic ensemble does not depend on offline configurations. Instead, it exploits the spatiotemporal patterns embedded in the training data to generate dynamic weights for the underlying weak distributed deep learners to create a stronger learner. Our evaluation experiments using three real-world datasets in the context of the smart city show that our proposed dynamic ensemble strategy leads to an improved error rate of up to 33% compared to the baseline strategy even when using $\frac {1}{3}$ of the training data. Moreover, using only 20% of the training data, the error rate of the model slightly increased by up to 2 in terms of mean square error. This increase is 82% less than the 11.3 increase seen in the baseline model. Therefore, our approach contributes to the reduced network traffic while not hindering the accuracy significantly.

[1]  W. Marsden I and J , 2012 .

[2]  Robert E. Schapire,et al.  The strength of weak learnability , 1990, Mach. Learn..

[3]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[4]  Jasbir S. Arora,et al.  Survey of multi-objective optimization methods for engineering , 2004 .

[5]  R. Schapire The Strength of Weak Learnability , 1990, Machine Learning.

[6]  Nicholas G. Polson,et al.  Deep learning for short-term traffic flow prediction , 2016, 1604.04527.

[7]  Andrés Ortiz,et al.  Ensembles of Deep Learning Architectures for the Early Diagnosis of the Alzheimer's Disease , 2016, Int. J. Neural Syst..

[8]  James Bailey,et al.  Traffic forecasting in complex urban networks: Leveraging big data and machine learning , 2015, 2015 IEEE International Conference on Big Data (Big Data).

[9]  H. T. Kung,et al.  Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices , 2017, 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS).

[10]  Marc'Aurelio Ranzato,et al.  Large Scale Distributed Deep Networks , 2012, NIPS.

[11]  Maguelonne Teisseire,et al.  Mining spatio-temporal data , 2006, Journal of Intelligent Information Systems.

[12]  Cha Zhang,et al.  Ensemble Machine Learning: Methods and Applications , 2012 .

[13]  Qinghua Zheng,et al.  Simple to Complex Cross-modal Learning to Rank , 2017, Comput. Vis. Image Underst..

[14]  Anthony K. H. Tung,et al.  SINGA: A Distributed Deep Learning Platform , 2015, ACM Multimedia.

[15]  Gang Wang,et al.  Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Mohsen Guizani,et al.  Deep Learning for IoT Big Data and Streaming Analytics: A Survey , 2017, IEEE Communications Surveys & Tutorials.

[17]  Ala I. Al-Fuqaha,et al.  Path Planning in Support of Smart Mobility Applications Using Generative Adversarial Networks , 2018, 2018 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData).

[18]  Mohsen Guizani,et al.  Semisupervised Deep Reinforcement Learning in Support of IoT and Smart City Services , 2018, IEEE Internet of Things Journal.

[19]  Arwa Alrawais,et al.  Fog Computing for the Internet of Things: Security and Privacy Issues , 2017, IEEE Internet Computing.

[20]  Yitao Liu,et al.  Deep learning based ensemble approach for probabilistic wind power forecasting , 2017 .

[21]  T. L. Vincent,et al.  Optimality in parametric systems , 1981 .

[22]  Giacomo Capizzi,et al.  Advanced and Adaptive Dispatch for Smart Grids by Means of Predictive Models , 2018, IEEE Transactions on Smart Grid.

[23]  Sasu Tarkoma,et al.  Theory and Practice of Bloom Filters for Distributed Systems , 2012, IEEE Communications Surveys & Tutorials.

[24]  Vincent W. S. Wong,et al.  Advanced Demand Side Management for the Future Smart Grid Using Mechanism Design , 2012, IEEE Transactions on Smart Grid.

[25]  Viktor K. Prasanna,et al.  On Using Cloud Platforms in a Software Architecture for Smart Energy Grids , 2010 .

[26]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[27]  Feng Gao,et al.  CityBench: A Configurable Benchmark to Evaluate RSP Engines Using Smart City Datasets , 2015, SEMWEB.

[28]  Min Chen,et al.  Enhanced Fingerprinting and Trajectory Prediction for IoT Localization in Smart Buildings , 2016, IEEE Transactions on Automation Science and Engineering.

[29]  Fei-Yue Wang,et al.  Traffic Flow Prediction With Big Data: A Deep Learning Approach , 2015, IEEE Transactions on Intelligent Transportation Systems.

[30]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[31]  Xiantong Zhen,et al.  Deep Ensemble Machine for Video Classification , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Irena Koprinska,et al.  Correlation and instance based feature selection for electricity load forecasting , 2015, Knowl. Based Syst..

[33]  Forrest N. Iandola,et al.  FireCaffe: Near-Linear Acceleration of Deep Neural Network Training on Compute Clusters , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Ponnuthurai Nagaratnam Suganthan,et al.  Empirical Mode Decomposition based ensemble deep learning for load demand time series forecasting , 2017, Appl. Soft Comput..

[35]  Ala I. Al-Fuqaha,et al.  Enabling Cognitive Smart Cities Using Big Data and Machine Learning: Approaches and Challenges , 2018, IEEE Communications Magazine.

[36]  Le Zhang,et al.  Ensemble deep learning for regression and time series forecasting , 2014, 2014 IEEE Symposium on Computational Intelligence in Ensemble Learning (CIEL).

[37]  Andrei Broder,et al.  Network Applications of Bloom Filters: A Survey , 2004, Internet Math..

[38]  R. Stephenson A and V , 1962, The British journal of ophthalmology.

[39]  Zhihui Li,et al.  Beyond Trace Ratio: Weighted Harmonic Mean of Trace Ratios for Multiclass Discriminant Analysis , 2017, IEEE Transactions on Knowledge and Data Engineering.