A Deep Reinforcement Learning Approach for Composing Moving IoT Services

We develop a novel framework for efficiently and effectively discovering crowdsourced services that move in close proximity to a user over a period of time. We introduce a moving crowdsourced service model which is modelled as a moving region. We propose a deep reinforcement learning-based composition approach to select and compose moving IoT services considering quality parameters. Additionally, we develop a parallel flock-based service discovery algorithm as a ground-truth to measure the accuracy of the proposed approach. The experiments on two real-world datasets verify the effectiveness and efficiency of the deep reinforcement learning-based approach.

[1]  Yucong Duan,et al.  Energy-Aware Service Composition of Configurable IoT Smart Things , 2018, 2018 14th International Conference on Mobile Ad-Hoc and Sensor Networks (MSN).

[2]  John Zimmerman,et al.  Swarthmore College , 2012 .

[3]  Haiyan Zhao,et al.  A Multi-agent Learning Model for Service Composition , 2012, 2012 IEEE Asia-Pacific Services Computing Conference.

[4]  M. Hansen,et al.  Participatory Sensing , 2019, Internet of Things.

[5]  Michiel C.J. Bliemer,et al.  Rewarding for Avoiding the Peak Period: A Synthesis of Four Studies in the Netherlands , 2010 .

[6]  Athman Bouguettaya,et al.  Incentive-Based Crowdsourcing of Hotspot Services , 2019, TOIT.

[7]  Quan Z. Sheng,et al.  On personalized cloud service provisioning for mobile users using adaptive and context-aware service composition , 2018, Computing.

[8]  Marco Saerens,et al.  Dynamic Web Service Composition within a Service-Oriented Architecture , 2007, IEEE International Conference on Web Services (ICWS 2007).

[9]  Zhaohui Wu,et al.  Mobility-Enabled Service Selection for Composite Services , 2016, IEEE Transactions on Services Computing.

[10]  Lei Chen,et al.  Spatial crowdsourcing: a survey , 2019, The VLDB Journal.

[11]  Minjie Zhang,et al.  Multi-Objective Service Composition Using Reinforcement Learning , 2013, ICSOC.

[12]  Stefan Müller Arisona,et al.  A Crowdsourcing Urban Simulation Platform on Smartphone Technology: Strategies for Urban Data Visualization and Transportation Mode Detection , 2012 .

[13]  Dong Zhao,et al.  CrowdOLR: Toward Object Location Recognition With Crowdsourced Fingerprints Using Smartphones , 2017, IEEE Transactions on Human-Machine Systems.

[14]  Albert Y. Zomaya,et al.  Mobility-Aware Service Selection in Mobile Edge Computing Systems , 2019, 2019 IEEE International Conference on Web Services (ICWS).

[15]  Jian Tang,et al.  Sensing as a service: A cloud computing system for mobile phone sensing , 2012, 2012 IEEE Sensors.

[16]  Lu Liu,et al.  Energy-aware composition for wireless sensor networks as a service , 2018, Future Gener. Comput. Syst..

[17]  Gilberto Marzano,et al.  Crowdsourcing solutions for supporting urban mobility , 2019 .

[18]  Steven D. Levitt,et al.  Using Big Data to Estimate Consumer Surplus: The Case of Uber , 2016 .

[19]  MengChu Zhou,et al.  Mobility-Aware Service Composition in Mobile Communities , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[20]  Abdelkarim Erradi,et al.  A service computing manifesto: the next 10 years , 2017, Commun. ACM.

[21]  Jiawei Han,et al.  Swarm: Mining Relaxed Temporal Moving Object Clusters , 2010, Proc. VLDB Endow..

[22]  Min Wang,et al.  Efficient Multi-way Theta-Join Processing Using MapReduce , 2012, Proc. VLDB Endow..

[23]  Jun Sun,et al.  CrowdService: Serving the individuals through mobile crowdsourcing and service composition , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[24]  Athman Bouguettaya,et al.  Composing Energy Services in a Crowdsourced IoT Environment , 2020 .

[25]  Jing Yuan,et al.  On Discovery of Traveling Companions from Streaming Trajectories , 2012, 2012 IEEE 28th International Conference on Data Engineering.

[26]  H. T. Mouftah,et al.  Sensing services in cloud-centric Internet of Things: A survey, taxonomy and challenges , 2015, 2015 IEEE International Conference on Communication Workshop (ICCW).

[27]  Reynold Cheng,et al.  Scalable Algorithms for Nearest-Neighbor Joins on Big Trajectory Data , 2016, IEEE Transactions on Knowledge and Data Engineering.

[28]  Jun Sun,et al.  CrowdService: Optimizing Mobile Crowdsourcing and Service Composition , 2018, ACM Trans. Internet Techn..

[29]  Petko Bakalov,et al.  On-line discovery of flock patterns in spatio-temporal data , 2009, GIS.

[30]  Joachim Gudmundsson,et al.  Computing longest duration flocks in trajectory data , 2006, GIS '06.

[31]  John Zimmerman,et al.  Mobile Transit Information from Universal Design and Crowdsourcing , 2011 .

[32]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[33]  Panos Kalnis,et al.  On Discovering Moving Clusters in Spatio-temporal Data , 2005, SSTD.

[34]  Abdelkarim Erradi,et al.  Mobile Crowdsourced Sensors Selection for Journey Services , 2018, ICSOC.

[35]  Cyrus Shahabi,et al.  GeoCrowd: enabling query answering with spatial crowdsourcing , 2012, SIGSPATIAL/GIS.

[36]  Hao Wang,et al.  Min-Max Planning of Time-Sensitive and Heterogeneous Tasks in Mobile Crowd Sensing , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[37]  Leandros Tassiulas,et al.  Enabling crowd-sourced mobile Internet access , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[38]  Hongbing Wang,et al.  Preference-Aware Web Service Composition by Reinforcement Learning , 2008, 2008 20th IEEE International Conference on Tools with Artificial Intelligence.

[39]  LimEe-Peng,et al.  Efficient mining of group patterns from user movement data , 2006 .

[40]  Timos K. Sellis,et al.  Spatio-Temporal Composition of Crowdsourced Services , 2015, ICSOC.

[41]  Hojung Cha,et al.  Automatically characterizing places with opportunistic crowdsensing using smartphones , 2012, UbiComp.

[42]  Alan Borning,et al.  OneBusAway: results from providing real-time arrival information for public transit , 2010, CHI.

[43]  James Bailey,et al.  Efficient mining of platoon patterns in trajectory databases , 2015, Data Knowl. Eng..

[44]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[45]  I. K. Altinel,et al.  Binary integer programming formulation and heuristics for differentiated coverage in heterogeneous sensor networks , 2008, Comput. Networks.

[46]  François Fouss,et al.  Optimal Tuning of Continual Online Exploration in Reinforcement Learning , 2006, ICANN.

[47]  Athman Bouguettaya,et al.  Crowdsourced Coverage as a Service: Two-Level Composition of Sensor Cloud Services , 2017, IEEE Transactions on Knowledge and Data Engineering.

[48]  Hongbing Wang,et al.  A Novel Approach to Large-Scale Services Composition , 2013, APWeb.

[49]  Zibin Zheng,et al.  Adaptive and Dynamic Service Composition via Multi-agent Reinforcement Learning , 2014, 2014 IEEE International Conference on Web Services.

[50]  Burak Kantarci,et al.  A reference model for crowdsourcing as a service , 2015, 2015 IEEE 4th International Conference on Cloud Networking (CloudNet).

[51]  Araz Taeihagh,et al.  Crowdsourcing: a new tool for policy-making? , 2017, Policy Sciences.

[52]  Chen Li,et al.  Efficient parallel set-similarity joins using MapReduce , 2010, SIGMOD Conference.

[53]  Yida Wang,et al.  Efficient mining of group patterns from user movement data , 2006, Data Knowl. Eng..

[54]  Athman Bouguettaya,et al.  Crowdsourcing Energy as a Service , 2018, ICSOC.

[55]  Meikang Qiu,et al.  Reinforcement Learning-based Content-Centric Services in Mobile Sensing , 2018, IEEE Network.

[56]  Pijush Kanti Dutta Pramanik,et al.  Mobility-aware service provisioning for delay tolerant applications in a mobile crowd computing environment , 2020 .

[57]  Jian Tang,et al.  Sensing as a Service: Challenges, Solutions and Future Directions , 2013, IEEE Sensors Journal.

[58]  Mahbub Hassan,et al.  A Survey of Wearable Devices and Challenges , 2017, IEEE Communications Surveys & Tutorials.

[59]  Huayu Wu,et al.  A General and Parallel Platform for Mining Co-Movement Patterns over Large-scale Trajectories , 2016, Proc. VLDB Endow..

[60]  Christian S. Jensen,et al.  Discovery of convoys in trajectory databases , 2008, Proc. VLDB Endow..

[61]  Xiaolong Xu,et al.  A Framework of Loose Travelling Companion Discovery from Human Trajectories , 2018, IEEE Transactions on Mobile Computing.

[62]  Murat Demirbas,et al.  Crowdsourcing location-based queries , 2011, 2011 IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops).