Hypothesis Transfer Learning for Efficient Data Computing in Smart Cities Environments

It is commonly assumed that in a smart city there will be thousands of mostly mobile/wireless smart devices (e.g. sensors, smart-phones, etc.) that will continuously generate big amounts of data. Data will have to be collected and processed in order to extract knowledge out of it, to feed users' and smart city applications. A typical approach to process such big amounts of data is to i) gather all the collected data on the cloud through wireless pervasive networks, and ii) perform data analysis operations exploiting machine learning techniques. However, according to many studies, this centralised cloud-based approach may not be sustainable from a networking point of view. The joint effect of data-intensive users' multimedia applications and smart cities monitoring and control applications may result in severe network congestions making applications hardly usable. To cope with this problem, in this paper we propose a distributed machine learning approach that does not require to move data in a centralised cloud platform, but processes it directly where it is collected. Specifically, we exploit Hypothesis Transfer Learning (HTL) to build a distributed machine learning framework. In our framework we train a series of partial models, each ''residing'' in a location where a subset of the dataset is generated. We then refine the partial models by exchanging them between locations, thus obtaining a unique complete model. Using an activity classification task on a reference dataset as a concrete example, we show that the classification accuracy of the HTL model is comparable with that of a model built out of the complete dataset, but the cost in term of network overhead is dramatically reduced. We then perform a sensitiveness analysis to characterise how the overhead depends on key parameters. It is also worth noticing that the HTL approach is suitable for applications dealing with privacy sensitive data, as data can stay where they are generated, and do not need to be transferred to third parties, i.e., to a cloud provider, to extract knowledge out of it.

[1]  Davide Anguita,et al.  Transition-Aware Human Activity Recognition Using Smartphones , 2016, Neurocomputing.

[2]  Martin Hasler,et al.  Distributed machine learning in networks by consensus , 2014, Neurocomputing.

[3]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[4]  Jason Weston,et al.  Fast Kernel Classifiers with Online and Active Learning , 2005, J. Mach. Learn. Res..

[5]  Emilio Parrado-Hernández,et al.  Distributed support vector machines , 2006, IEEE Trans. Neural Networks.

[6]  Eleonora Borgia,et al.  The Internet of Things vision: Key features, applications and open issues , 2014, Comput. Commun..

[7]  Teruo Higashino,et al.  Edge-centric Computing: Vision and Challenges , 2015, CCRV.

[8]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[9]  Yoram Singer,et al.  Using and combining predictors that specialize , 1997, STOC '97.

[10]  Alberto L. Sangiovanni-Vincentelli Let's get physical: Adding physical dimensions to cyber systems , 2015, ISLPED.

[11]  Marco Conti,et al.  From opportunistic networks to opportunistic computing , 2010, IEEE Communications Magazine.

[12]  Tao Wang,et al.  Deep learning with COTS HPC systems , 2013, ICML.

[13]  Dianhui Wang,et al.  Distributed learning for Random Vector Functional-Link networks , 2015, Inf. Sci..

[14]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[15]  Ilja Kuzborskij,et al.  Transfer Learning Through Greedy Subset Selection , 2014, ICIAP.

[16]  Marc'Aurelio Ranzato,et al.  Large Scale Distributed Deep Networks , 2012, NIPS.