Layerwise Geo-Distributed Computing between Cloud and IoT

In this paper, we propose a novel architecture for a deep learning system, named k-degree layer-wise network, to realize efficient geo-distributed computing between Cloud and Internet of Things (IoT). The geo-distributed computing extends Cloud to the geographical verge of the network in the neighbor of IoT. The basic ideas of the proposal include a k-degree constraint and a layer-wise constraint. The k-degree constraint is defined such that the degree of each vertex on the h layer is exactly k to extend the existing deep belief networks and control the communication cost. The layer-wise constraint is defined such that the layer-wise degrees are monotonically decreasing in positive direction to gradually reduce the dimension of data. We prove the k-degree layer-wise network is sparse, while a typical deep neural network is dense. In an evaluation on the M-distributed MNIST database, the proposal is superior to a state-of-the-art model in terms of communication cost and learning time with scalability.

[1]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[2]  Xue-wen Chen,et al.  Large-Scale Deep Belief Nets With MapReduce , 2014, IEEE Access.

[3]  Malek Ben Salem,et al.  Fog Computing: Mitigating Insider Data Theft Attacks in the Cloud , 2012, 2012 IEEE Symposium on Security and Privacy Workshops.

[4]  Zhen Xiao,et al.  Dynamic Resource Allocation Using Virtual Machines for Cloud Computing Environment , 2013, IEEE Transactions on Parallel and Distributed Systems.

[5]  In-Jung Lee,et al.  Big data processing framework of road traffic collision using distributed CEP , 2014, The 16th Asia-Pacific Network Operations and Management Symposium.

[6]  Mohamed Cheriet,et al.  Deep-Belief-Network Based Rescoring Approach for Handwritten Word Recognition , 2014, 2014 14th International Conference on Frontiers in Handwriting Recognition.

[7]  Petr Fousek,et al.  Factorial Hidden Restricted Boltzmann Machines for noise robust speech recognition , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[9]  Xiaohua Jia,et al.  An Efficient and Secure Dynamic Auditing Protocol for Data Storage in Cloud Computing , 2013, IEEE Transactions on Parallel and Distributed Systems.

[10]  Geoffrey Zweig,et al.  Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[11]  Yiqiang Sheng,et al.  Scalable Intelligence-Enabled Networking With Traffic Engineering in 5G Scenarios for Future Audio-Visual-Tactile Internet , 2019, IEEE Access.

[12]  Chun Yang,et al.  Shallow Classification or Deep Learning: An Experimental Study , 2014, 2014 22nd International Conference on Pattern Recognition.

[13]  Zhili Sun,et al.  A Sevice-Oriented Broker for Bulk Data Transfer in Cloud Computing , 2010, 2010 Ninth International Conference on Grid and Cloud Computing.

[14]  Hideki Tode,et al.  A generalized spatial boundary analysis method for clustering/multi-hop hybrid routing in Wireless Sensor Networks , 2011, 2011 IEEE GLOBECOM Workshops (GC Wkshps).

[15]  Kurt Rothermel,et al.  MigCEP: operator migration for mobility driven distributed complex event processing , 2013, DEBS.

[16]  Ivan Stojmenovic,et al.  The Fog computing paradigm: Scenarios and security issues , 2014, 2014 Federated Conference on Computer Science and Information Systems.

[17]  Brian Kingsbury,et al.  New types of deep neural network learning for speech recognition and related applications: an overview , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[18]  Rui Wang,et al.  Bridging Data in the Clouds: An Environment-Aware System for Geographically Distributed Data Transfers , 2014, 2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[19]  Joseph M. Hellerstein,et al.  Distributed GraphLab: A Framework for Machine Learning in the Cloud , 2012, Proc. VLDB Endow..

[20]  Yann LeCun,et al.  Convolutional networks and applications in vision , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[21]  Roman Y. Shtykh,et al.  Distributed Data Stream Processing with Onix , 2014, 2014 IEEE Fourth International Conference on Big Data and Cloud Computing.

[22]  Masaya Yoshida,et al.  Deployment adviser tool for wireless sensor networks , 2014, 2014 IEEE World Forum on Internet of Things (WF-IoT).

[23]  Paolo Bellavista,et al.  Priority-Based Resource Scheduling in Distributed Stream Processing Systems for Big Data Applications , 2014, 2014 IEEE/ACM 7th International Conference on Utility and Cloud Computing.