A Class of R*-tree Indexes for Spatial-Visual Search of Geo-tagged Street Images

Due to the prevalence of GPS-equipped cameras (e.g., smartphones and surveillance cameras), massive amounts of geo-tagged images capturing urban streets are increasingly being collected. Consequently, many smart city applications have emerged, relying on efficient image search. Such searches include spatial-visual queries in which spatial and visual properties are used in tandem to retrieve similar images to a given query image within a given geographical region. Towards this end, new index structures that organize images based on both spatial and visual properties are needed to efficiently execute such queries. Based on our observation that street images are typically similar in the same spatial locality, index structures for spatial-visual queries can be effectively built on a spatial index (i.e., R*-tree). Therefore, we propose a class of R*-tree indexes, particularly, by associating each node with two separate minimum bounding rectangles (MBR), one for spatial and the other for (dimension-reduced) visual properties of their contained images, and adapting the R*-tree optimization criteria to both property types.

[1]  Dimitrios Skoutas,et al.  Indexing Geolocated Time Series Data , 2017, SIGSPATIAL/GIS.

[2]  Dimitrios Skoutas,et al.  Visual Exploration of Geolocated Time Series with Hybrid Indexing , 2019, Big Data Res..

[3]  Mubarak Shah,et al.  Image Geo-Localization Based on MultipleNearest Neighbor Feature Matching UsingGeneralized Graphs , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Cyrus Shahabi,et al.  Recognizing Material of a Covered Object: A Case Study With Graffiti , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[5]  Cyrus Shahabi,et al.  Hybrid Indexes for Spatial-Visual Search , 2017, ACM Multimedia.

[6]  Chih-Ya Shen,et al.  On socio-spatial group query for location-based social networks , 2012, KDD.

[7]  Cyrus Shahabi,et al.  GeoUGV: user-generated mobile video dataset with fine granularity spatial metadata , 2016, MMSys.

[8]  Cyrus Shahabi,et al.  A Data-Centric Approach for Image Scene Localization , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[9]  Ronan Sicre,et al.  Particular object retrieval with integral max-pooling of CNN activations , 2015, ICLR.

[10]  Cyrus Shahabi,et al.  A Deep Learning Approach for Road Damage Detection from Smartphone Images , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[11]  Cyrus Shahabi,et al.  Efficient indexing and retrieval of large-scale geo-tagged video databases , 2016, GeoInformatica.

[12]  Piotr Indyk,et al.  Approximate nearest neighbors: towards removing the curse of dimensionality , 1998, STOC '98.

[13]  Christian S. Jensen,et al.  Spatial Keyword Query Processing: An Experimental Evaluation , 2013, Proc. VLDB Endow..

[14]  Hans-Peter Kriegel,et al.  The R*-tree: an efficient and robust access method for points and rectangles , 1990, SIGMOD '90.

[15]  Cyrus Shahabi,et al.  Image Classification to Determine the Level of Street Cleanliness: A Case Study , 2018, 2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM).