Efficient region of visual interests search for geo-multimedia data

With the proliferation of online social networking services and mobile smart devices equipped with mobile communications module and position sensor module, massive amount of multimedia data has been collected, stored and shared. This trend has put forward higher request on massive multimedia data retrieval. In this paper, we investigate a novel spatial query named region of visual interests query (RoVIQ), which aims to search users containing geographical information and visual words. Three baseline methods are presented to introduce how to exploit existing techniques to address this problem. Then we propose the definition of this query and related notions at the first time. To improve the performance of query, we propose a novel spatial indexing structure called quadtree based inverted visual index which is a combination of quadtree, inverted index and visual words. Based on it, we design a efficient search algorithm named region of visual interests search to support RoVIQ. Experimental evaluations on real geo-image datasets demonstrate that our solution outperforms state-of-the-art method.

[1]  Antonin Guttman,et al.  R-trees: a dynamic index structure for spatial searching , 1984, SIGMOD '84.

[2]  Lin Wu,et al.  Deep adaptive feature embedding with local sample distributions for person re-identification , 2017, Pattern Recognit..

[3]  Lin Wu,et al.  Shifting Hypergraphs by Probabilistic Voting , 2014, PAKDD.

[4]  Jun Hu,et al.  SEAL: Spatio-Textual Similarity Search , 2012, Proc. VLDB Endow..

[5]  Lin Wu,et al.  Iterative Views Agreement: An Iterative Low-Rank Based Structured Optimization Method to Multi-View Spectral Clustering , 2016, IJCAI.

[6]  Linda G. Shapiro,et al.  A SIFT descriptor with global context , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[7]  Htoo Htet Aung,et al.  Efficient continuous top-k spatial keyword queries on road networks , 2014, GeoInformatica.

[8]  Xue Li,et al.  Deep Attention-Based Spatially Recursive Networks for Fine-Grained Visual Recognition , 2019, IEEE Transactions on Cybernetics.

[9]  Yu Liu,et al.  Multi-focus image fusion with dense SIFT , 2015, Inf. Fusion.

[10]  Ling Shao,et al.  Cycle-Consistent Deep Generative Hashing for Cross-Modal Retrieval , 2018, IEEE Transactions on Image Processing.

[11]  Lin Wu,et al.  Robust Subspace Clustering for Multi-View Data by Exploiting Correlation Consensus , 2015, IEEE Transactions on Image Processing.

[12]  Ricardo da Silva Torres,et al.  Color and texture applied to a signature-based bag of visual words method for image retrieval , 2017, Multimedia Tools and Applications.

[13]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[14]  Lin Wu,et al.  Beyond Low-Rank Representations: Orthogonal Clustering Basis Reconstruction with Optimized Graph Structure for Multi-view Spectral Clustering , 2017, Neural Networks.

[15]  Kenneth Steiglitz,et al.  Operations on Images Using Quad Trees , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Lin Wu,et al.  Robust Hashing for Multi-View Data: Jointly Learning Low-Rank Kernelized Similarity Consensus and Hash Functions , 2016, Image Vis. Comput..

[17]  Christos Faloutsos,et al.  Multiattribute hashing using Gray codes , 1986, SIGMOD '86.

[18]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[19]  Lin Wu,et al.  Effective Multi-Query Expansions: Robust Landmark Retrieval , 2015, ACM Multimedia.

[20]  Naphtali Rishe,et al.  Keyword Search on Spatial Databases , 2008, 2008 IEEE 24th International Conference on Data Engineering.

[21]  Shumeet Baluja,et al.  VisualRank: Applying PageRank to Large-Scale Image Search , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Savvas A. Chatzichristofis,et al.  Image moment invariants as local features for content based image retrieval using the Bag-of-Visual-Words model , 2015, Pattern Recognit. Lett..

[23]  Yan Ke,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[24]  Beng Chin Ooi,et al.  Collective spatial keyword querying , 2011, SIGMOD '11.

[25]  Xuemin Lin,et al.  AP-Tree: Efficiently support continuous spatial-keyword queries over stream , 2015, 2015 IEEE 31st International Conference on Data Engineering.

[26]  Yuan Tian,et al.  ROAD: A New Spatial Object Search Framework for Road Networks , 2012, IEEE Transactions on Knowledge and Data Engineering.

[27]  Tao Guo,et al.  Efficient Algorithms for Answering the m-Closest Keywords Query , 2015, SIGMOD Conference.

[28]  Lin Wu,et al.  Multiview Spectral Clustering via Structured Low-Rank Matrix Factorization , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Christian S. Jensen,et al.  Spatial Keyword Querying , 2012, ER.

[30]  Jiajie Xu,et al.  Interactive Top-k Spatial Keyword queries , 2015, 2015 IEEE 31st International Conference on Data Engineering.

[31]  Christian S. Jensen,et al.  Efficient Retrieval of the Top-k Most Relevant Spatial Web Objects , 2009, Proc. VLDB Endow..

[32]  Yang Wang,et al.  Structured Deep Hashing with Convolutional Neural Networks for Fast Person Re-identification , 2017, Comput. Vis. Image Underst..

[33]  Yuping Zhang,et al.  MBR-SIFT: A mirror reflected invariant feature descriptor using a binary representation for image matching , 2017, PloS one.

[34]  Lin Wu,et al.  Where-and-When to Look: Deep Siamese Attention Networks for Video-Based Person Re-Identification , 2018, IEEE Transactions on Multimedia.

[35]  Muhammad Aamir Cheema,et al.  Diversified Spatial Keyword Search On Road Networks , 2014, EDBT.

[36]  Lin Wu,et al.  Efficient image and tag co-ranking: a bregman divergence optimization method , 2013, ACM Multimedia.

[37]  João B. Rocha-Junior,et al.  Top-k spatial keyword queries on road networks , 2012, EDBT '12.

[38]  Adel M. Alimi,et al.  Bimodal biometric system for hand shape and palmprint recognition based on SIFT sparse representation , 2017, Multimedia Tools and Applications.

[39]  João B. Rocha-Junior,et al.  Efficient Processing of Top-k Spatial Keyword Queries , 2011, SSTD.

[40]  Andrew Dawson,et al.  Suicide prevention through means restriction: Impact of the 2008-2011 pesticide restrictions on suicide in Sri Lanka , 2017, PloS one.

[41]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[42]  Jong-Uk Hou,et al.  A SIFT features based blind watermarking for DIBR 3D images , 2018, Multimedia Tools and Applications.

[43]  Lin Wu,et al.  Exploiting Correlation Consensus: Towards Subspace Clustering for Multi-modal Data , 2014, ACM Multimedia.

[44]  Lin Wu,et al.  Effective Multi-Query Expansions: Collaborative Deep Networks for Robust Landmark Retrieval , 2017, IEEE Transactions on Image Processing.

[45]  Lin Wu,et al.  LBMCH: Learning Bridging Mapping for Cross-modal Hashing , 2015, SIGIR.

[46]  Ji Wan,et al.  Deep Learning for Content-Based Image Retrieval: A Comprehensive Study , 2014, ACM Multimedia.

[47]  Yang Wang,et al.  Towards metric fusion on multi-view data: a cross-view based graph random walk approach , 2013, CIKM.

[48]  Xuemin Lin,et al.  Inverted Linear Quadtree: Efficient Top K Spatial Keyword Search , 2016, IEEE Transactions on Knowledge and Data Engineering.

[49]  Anthony K. H. Tung,et al.  Scalable top-k spatial keyword search , 2013, EDBT '13.

[50]  Hans-Peter Kriegel,et al.  The R*-tree: an efficient and robust access method for points and rectangles , 1990, SIGMOD '90.

[51]  Irene Gargantini,et al.  An effective way to represent quadtrees , 1982, CACM.

[52]  Ken C. K. Lee,et al.  IR-Tree: An Efficient Index for Geographic Document Search , 2011, IEEE Trans. Knowl. Data Eng..

[53]  Fangyuan Wang,et al.  Large Scale Image Retrieval with Practical Spatial Weighting for Bag-of-Visual-Words , 2013, MMM.

[54]  Nicu Sebe,et al.  Content-based multimedia information retrieval: State of the art and challenges , 2006, TOMCCAP.

[55]  Lin Wu,et al.  Unsupervised Metric Fusion Over Multiview Data by Graph Random Walk-Based Cross-View Diffusion , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[56]  Xin Li,et al.  Best Keyword Cover Search , 2015, IEEE Transactions on Knowledge and Data Engineering.

[57]  Chen Li,et al.  Processing Spatial-Keyword (SK) Queries in Geographic Information Retrieval (GIR) Systems , 2007, 19th International Conference on Scientific and Statistical Database Management (SSDBM 2007).

[58]  Lin Wu,et al.  What-and-Where to Match: Deep Spatially Multiplicative Integration Networks for Person Re-identification , 2017, Pattern Recognit..