An Edge Computing-Based Photo Crowdsourcing Framework for Real-Time 3D Reconstruction

Image-based three-dimensional (3D) reconstruction utilizes a set of photos to build 3D model and can be widely used in many emerging applications such as augmented reality (AR) and disaster recovery. Most of existing 3D reconstruction methods require a mobile user to walk around the target area and reconstruct objectives with a hand-held camera, which is inefficient and time-consuming. To meet the requirements of delay intensive and resource hungry applications in 5G, we propose an edge computing-based photo crowdsourcing (EC-PCS) framework in this paper. The main objective is to collect a set of representative photos from ubiquitous mobile and Internet of Things (IoT) devices at the network edge for real-time 3D model reconstruction, with network resource and monetary cost considerations. Specifically, we first propose a photo pricing mechanism by jointly considering their freshness, resolution and data size. Then, we design a novel photo selection scheme to dynamically select a set of photos with the required target coverage and the minimum monetary cost. We prove the NP-hardness of such problem, and develop an efficient greedy-based approximation algorithm to obtain a near-optimal solution. Moreover, an optimal network resource allocation scheme is presented, in order to minimize the maximum uploading delay of the selected photos to the edge server. Finally, a 3D reconstruction algorithm and a 3D model caching scheme are performed by the edge server in real time. Extensive experimental results based on real-world datasets demonstrate the superior performance of our EC-PCS system over the existing mechanisms.

[1]  Mohammed Bennamoun,et al.  Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  H. Vincent Poor,et al.  Machine Intelligence at the Edge With Learning Centric Power Allocation , 2020, IEEE Transactions on Wireless Communications.

[3]  Paolo Cignoni,et al.  MeshLab: an Open-Source Mesh Processing Tool , 2008, Eurographics Italian Chapter Conference.

[4]  Xing Xie,et al.  PicPick: a generic data selection framework for mobile crowd photography , 2016, Personal and Ubiquitous Computing.

[5]  Hugh F. Durrant-Whyte,et al.  Simultaneous localization and mapping: part I , 2006, IEEE Robotics & Automation Magazine.

[6]  Matti Latva-aho,et al.  Caching in Wireless Small Cell Networks: A Storage-Bandwidth Tradeoff , 2016, IEEE Communications Letters.

[7]  Torsten Sattler,et al.  3D Modeling on the Go: Interactive 3D Reconstruction of Large-Scale Scenes on Mobile Devices , 2015, 2015 International Conference on 3D Vision.

[8]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[9]  Marc Pollefeys,et al.  Live Metric 3D Reconstruction on Mobile Phones , 2013, 2013 IEEE International Conference on Computer Vision.

[10]  Ming Xu,et al.  From Uncertain Photos to Certain Coverage: a Novel Photo Selection Approach to Mobile Crowdsensing , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[11]  Jane J. Ye,et al.  Enhanced Karush–Kuhn–Tucker Conditions for Mathematical Programs with Equilibrium Constraints , 2014, J. Optim. Theory Appl..

[12]  Laurence A. Wolsey,et al.  An analysis of the greedy algorithm for the submodular set covering problem , 1982, Comb..

[13]  Wei-Ho Chung,et al.  Latency-Driven Cooperative Task Computing in Multi-user Fog-Radio Access Networks , 2017, 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS).

[14]  Zoran Popovic,et al.  PhotoCity: training experts at large-scale image acquisition through a competitive game , 2011, CHI.

[15]  Xu Chen,et al.  Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing , 2019, Proceedings of the IEEE.

[16]  Paulo Drews,et al.  Analyzing and exploring feature detectors in images , 2011, 2011 9th IEEE International Conference on Industrial Informatics.

[17]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[18]  Ming-Syan Chen,et al.  Deep Censored Learning of the Winning Price in the Real Time Bidding , 2018, KDD.

[19]  Shahrouz Yousefi,et al.  Analysis of the user experience in a 3D gesture-based supported mobile VR game , 2017, VRST.

[20]  Jianwei Huang,et al.  Delay-Sensitive Mobile Crowdsensing: Algorithm Design and Economics , 2018, IEEE Transactions on Mobile Computing.

[21]  Yi Wang,et al.  Photo crowdsourcing for area coverage in resource constrained environments , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[22]  George Karakostas,et al.  A better approximation ratio for the vertex cover problem , 2005, TALG.

[23]  Qi Han,et al.  Toward real-time and cooperative mobile visual sensing and sharing , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[24]  Dieter Schmalstieg,et al.  Augmented Reality Scouting for Interactive 3D Reconstruction , 2007, 2007 IEEE Virtual Reality Conference.

[25]  Jie Zhang,et al.  OFDMA femtocells: A roadmap on interference avoidance , 2009, IEEE Communications Magazine.

[26]  Jonathan Rodriguez,et al.  Robust Mobile Crowd Sensing: When Deep Learning Meets Edge Computing , 2018, IEEE Network.

[27]  Jeongho Kwak,et al.  DREAM: Dynamic Resource and Task Allocation for Energy Minimization in Mobile Cloud Systems , 2015, IEEE Journal on Selected Areas in Communications.