A Crowd-Based Image Learning Framework using Edge Computing for Smart City Applications

Smart city applications covering a wide area such as traffic monitoring and pothole detection are gradually adopting more image machine learning algorithms utilizing ubiquitous camera sensors. To support such applications, an edge computing paradigm focuses on processing large amount of multimedia data at the edge to offload processing cost and reduce long-distance traffic and latency. However, existing edge computing approaches rely on pre-trained static models and are limited in supporting diverse classes of edge devices as well as learning models to support them. This research proposes a novel crowd-based learning framework which allows edge devices with diverse resource capabilities to perform machine learning towards the realization of image-based smart city applications. The intelligent retraining algorithm allows sharing key visual features to achieve a higher accuracy based on the temporal and geospatial uniqueness. Our evaluation shows the trade-off between accuracy and the resource constraints of the edge devices, while the model re-sizing option enables running machine learning models on edge devices with high flexibility.

[1]  Mahadev Satyanarayanan,et al.  Balancing performance, energy, and quality in pervasive computing , 2002, Proceedings 22nd International Conference on Distributed Computing Systems.

[2]  G. Griffin,et al.  Caltech-256 Object Category Dataset , 2007 .

[3]  Cyrus Shahabi,et al.  Geo-Spatial Multimedia Sentiment Analysis in Disasters , 2017, 2017 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[4]  Ramesh Govindan,et al.  Odessa: enabling interactive perception applications on mobile devices , 2011, MobiSys '11.

[5]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Cyrus Shahabi,et al.  A Data-Centric Approach for Image Scene Localization , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[7]  Daren C. Brabham Crowdsourcing as a Model for Problem Solving , 2008 .

[8]  Nilesh Padhariya,et al.  Crowdlearning: An incentive-based learning platform for crowd , 2014, 2014 Seventh International Conference on Contemporary Computing (IC3).

[9]  Cyrus Shahabi,et al.  Image Classification to Determine the Level of Street Cleanliness: A Case Study , 2018, 2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM).

[10]  Wayne H. Wolf,et al.  Key frame selection by motion analysis , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[11]  Chee Sun Liew,et al.  Data-Intensive Workflow Optimization Based on Application Task Graph Partitioning in Heterogeneous Computing Systems , 2014, 2014 IEEE Fourth International Conference on Big Data and Cloud Computing.

[12]  Cyrus Shahabi,et al.  Recognizing Material of a Covered Object: A Case Study With Graffiti , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[13]  Rahul Sukthankar,et al.  SLIPstream: scalable low-latency interactive perception on streaming data , 2009, NOSSDAV '09.

[14]  Edward J. Delp,et al.  Automatic gang graffiti recognition and interpretation , 2017, J. Electronic Imaging.

[15]  Cyrus Shahabi,et al.  Task matching and scheduling for multiple workers in spatial crowdsourcing , 2015, SIGSPATIAL/GIS.

[16]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[17]  Shinpei Kato,et al.  GPUrpc: Exploring Transparent Access to Remote GPUs , 2016, ACM Trans. Embed. Comput. Syst..

[18]  Cyrus Shahabi,et al.  Efficient indexing and retrieval of large-scale geo-tagged video databases , 2016, GeoInformatica.

[19]  Enrique Saurez,et al.  Incremental deployment and migration of geo-distributed situation awareness applications in the fog , 2016, DEBS.

[20]  Cyrus Shahabi,et al.  TVDP: Translational Visual Data Platform for Smart Cities , 2019, 2019 IEEE 35th International Conference on Data Engineering Workshops (ICDEW).

[21]  Manish Parashar,et al.  Data-Driven Stream Processing at the Edge , 2017, 2017 IEEE 1st International Conference on Fog and Edge Computing (ICFEC).

[22]  Jörg Ott,et al.  Enabling Fine-Grained Edge Offloading for IoT , 2017, SIGCOMM Posters and Demos.

[23]  Cyrus Shahabi,et al.  Scalable Spatial Crowdsourcing: A Study of Distributed Algorithms , 2015, 2015 16th IEEE International Conference on Mobile Data Management.

[24]  Cyrus Shahabi,et al.  GeoCrowd: enabling query answering with spatial crowdsourcing , 2012, SIGSPATIAL/GIS.

[25]  Cyrus Shahabi,et al.  Gift: A geospatial image and video filtering tool for computer vision applications with geo-tagged mobile videos , 2015, 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[26]  Zhuo Chen,et al.  Edge Analytics in the Internet of Things , 2015, IEEE Pervasive Computing.

[27]  Cyrus Shahabi,et al.  Hybrid Indexes for Spatial-Visual Search , 2017, ACM Multimedia.

[28]  Cyrus Shahabi,et al.  MediaQ: mobile multimedia management system , 2014, MMSys '14.

[29]  Cyrus Shahabi,et al.  Real-Time Traffic Video Analysis Using Intel Viewmont Coprocessor , 2013, DNIS.

[30]  Roger Zimmermann,et al.  Viewable scene modeling for geospatial video search , 2008, ACM Multimedia.

[31]  Bhaskar Krishnamachari,et al.  I3: An IoT Marketplace for Smart Communities , 2018, MobiSys.

[32]  Mingyan Liu,et al.  Crowd Learning: Improving Online Decision Making Using Crowdsourced Data , 2017, IJCAI.

[33]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[34]  Christoph Mertz,et al.  1 CITY-WIDE ROAD DISTRESS MONITORING WITH SMARTPHONES , 2014 .

[35]  Cyrus Shahabi,et al.  A Deep Learning Approach for Road Damage Detection from Smartphone Images , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[36]  Mahadev Satyanarayanan,et al.  The case for cyber foraging , 2002, EW 10.

[37]  Cyrus Shahabi,et al.  Spatial Coverage Measurement of Geo- Tagged Visual Data: A Database Approach , 2018, 2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM).

[38]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.