A Semi-Supervised Method for Surveillance-Based Visual Location Recognition

In this paper, we are devoted to solving the problem of crossing surveillance and mobile phone visual location recognition, especially for the case that the query and reference images are captured by mobile phone and surveillance camera, respectively. Besides, we also study the influence of the environmental condition variations on this problem. To explore that problem, we first build a cross-device location recognition dataset, which includes images of 22 locations taken by mobile phones and surveillance cameras under different time and weather conditions. Then based on careful analysis of the problems existing in the data, we specifically design a method which unifies an unsupervised subspace alignment method and the semi-supervised Laplacian support vector machine. Experiments are performed on our dataset. Compared with several related methods, our method shows to be more efficient on the problem of crossing surveillance and mobile phone visual location recognition. Furthermore, the influence of several factors such as feature, time, and weather is studied.

[1]  Mubarak Shah,et al.  Image Geo-Localization Based on MultipleNearest Neighbor Feature Matching UsingGeneralized Graphs , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  S. Sathiya Keerthi,et al.  Optimization Techniques for Semi-Supervised Support Vector Machines , 2008, J. Mach. Learn. Res..

[3]  Alexei A. Efros,et al.  What Do the Sun and the Sky Tell Us About the Camera? , 2010, International Journal of Computer Vision.

[4]  C. Schmid,et al.  On the burstiness of visual elements , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[6]  Ivor W. Tsang,et al.  Large-Scale Sparsified Manifold Regularization , 2006, NIPS.

[7]  Alexei A. Efros,et al.  IM2GPS: estimating geographic information from a single image , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Yi Dong,et al.  Automatic Geolocation Correction of Satellite Imagery , 2015, International Journal of Computer Vision.

[9]  Riad I. Hammoud,et al.  Overhead-Based Image and Video Geo-localization Framework , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[10]  Xuelong Li,et al.  Flowing on Riemannian Manifold: Domain Adaptation by Shifting Covariance , 2014, IEEE Transactions on Cybernetics.

[11]  Nathan Halko,et al.  An Algorithm for the Principal Component Analysis of Large Data Sets , 2010, SIAM J. Sci. Comput..

[12]  Kaizhu Huang,et al.  Learning Locality Preserving Graph from Data , 2014, IEEE Transactions on Cybernetics.

[13]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[14]  Avideh Zakhor,et al.  Location-based image retrieval for urban environments , 2011, 2011 18th IEEE International Conference on Image Processing.

[15]  Mikhail Belkin,et al.  Beyond the point cloud: from transductive to semi-supervised learning , 2005, ICML.

[16]  Kristen Grauman,et al.  Reshaping Visual Datasets for Domain Adaptation , 2013, NIPS.

[17]  Avideh Zakhor,et al.  Reduced-complexity data acquisition system for image-based localization in indoor environments , 2013, International Conference on Indoor Positioning and Indoor Navigation.

[18]  Mubarak Shah,et al.  Accurate Image Localization Based on Google Maps Street View , 2010, ECCV.

[19]  Marc Pollefeys,et al.  Image Based Geo-localization in the Alps , 2016, International Journal of Computer Vision.

[20]  Avideh Zakhor,et al.  Indoor Localization Algorithms for an Ambulatory Human Operated 3D Mobile Mapping System , 2013, Remote. Sens..

[21]  Noah Snavely,et al.  Graph-Based Discriminative Learning for Location Recognition , 2013, International Journal of Computer Vision.

[22]  Chong Wang,et al.  Weakly Supervised Object Localization with Latent Category Learning , 2014, ECCV.

[23]  Cheng Wu,et al.  Semi-Supervised and Unsupervised Extreme Learning Machines , 2014, IEEE Transactions on Cybernetics.

[24]  Fadi Dornaika,et al.  Learning Flexible Graph-Based Semi-Supervised Embedding , 2016, IEEE Transactions on Cybernetics.

[25]  Mikhail Belkin,et al.  Laplacian Support Vector Machines Trained in the Primal , 2009, J. Mach. Learn. Res..

[26]  Thorsten Joachims,et al.  Transductive Learning via Spectral Graph Partitioning , 2003, ICML.

[27]  Daniel P. Huttenlocher,et al.  Location Recognition Using Prioritized Feature Matching , 2010, ECCV.

[28]  Tieniu Tan,et al.  Semi-supervised Learning for Cross-Device Visual Location Recognition , 2014, 2014 22nd International Conference on Pattern Recognition.

[29]  Tomás Pajdla,et al.  Avoiding Confusing Features in Place Recognition , 2010, ECCV.

[30]  Tieniu Tan,et al.  Uniform low-rank representation for unsupervised visual domain adaptation , 2015, 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR).

[31]  Anas Al-Nuaimi,et al.  Mobile Visual Location Recognition , 2013 .

[32]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[33]  Xin Chen,et al.  City-scale landmark identification on mobile devices , 2011, CVPR 2011.

[34]  Rongrong Ji,et al.  Active query sensing for mobile location search , 2011, ACM Multimedia.

[35]  Yi Dong,et al.  Geo-localization using Volumetric Representations of Overhead Imagery , 2015, International Journal of Computer Vision.

[36]  Tomás Pajdla,et al.  Learning and Calibrating Per-Location Classifiers for Visual Place Recognition , 2013, International Journal of Computer Vision.

[37]  Yuan Shi,et al.  Geodesic flow kernel for unsupervised domain adaptation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Trevor Darrell,et al.  Adapting Visual Category Models to New Domains , 2010, ECCV.

[39]  Bernd Girod,et al.  Mobile Visual Search , 2011, IEEE Signal Processing Magazine.

[40]  Kristen Grauman,et al.  Connecting the Dots with Landmarks: Discriminatively Learning Domain-Invariant Features for Unsupervised Domain Adaptation , 2013, ICML.

[41]  Trevor Darrell,et al.  Discovering Latent Domains for Multisource Domain Adaptation , 2012, ECCV.

[42]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[43]  Marc Pollefeys,et al.  Large Scale Visual Geo-Localization of Images in Mountainous Terrain , 2012, ECCV.

[44]  Dong Liu,et al.  Robust visual domain adaptation with low-rank reconstruction , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[46]  Avideh Zakhor,et al.  Image Based Localization in Indoor Environments , 2013, 2013 Fourth International Conference on Computing for Geospatial Research and Application.

[47]  Serge J. Belongie,et al.  Cross-View Image Geolocalization , 2013, CVPR.

[48]  Marc Pollefeys,et al.  Handling Urban Location Recognition as a 2D Homothetic Problem , 2010, ECCV.

[49]  Pascal Monasse,et al.  Global Multiple-View Color Consistency , 2013 .

[50]  Dani Lischinski,et al.  Optimizing color consistency in photo collections , 2013, ACM Trans. Graph..

[51]  Chong Wang,et al.  Cross-Domain Object Recognition Using Object Alignment , 2015, BMVC.

[52]  Mikhail Belkin,et al.  Using manifold structure for partially labelled classification , 2002, NIPS 2002.

[53]  Florian Schweiger,et al.  TUMindoor: An extensive image and point cloud dataset for visual indoor localization and mapping , 2012, 2012 19th IEEE International Conference on Image Processing.

[54]  Tinne Tuytelaars,et al.  Unsupervised Visual Domain Adaptation Using Subspace Alignment , 2013, 2013 IEEE International Conference on Computer Vision.

[55]  Richard Szeliski,et al.  City-Scale Location Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Ivor W. Tsang,et al.  Combating Negative Transfer From Predictive Distribution Differences , 2013, IEEE Transactions on Cybernetics.

[57]  Min Jiang,et al.  Integration of Global and Local Metrics for Domain Adaptation Learning Via Dimensionality Reduction , 2017, IEEE Transactions on Cybernetics.

[58]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[59]  Alexei A. Efros,et al.  What makes Paris look like Paris? , 2015, Commun. ACM.

[60]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[61]  Masatoshi Okutomi,et al.  Visual Place Recognition with Repetitive Structures , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Marc Pollefeys,et al.  IMAGE LOCALIZATION IN SATELLITE IMAGERY WITH FEATURE-BASED INDEXING , 2008 .

[63]  Mikhail Belkin,et al.  Linear Manifold Regularization for Large Scale Semi-supervised Learning , 2005 .