Building block level urban land-use information retrieval based on Google Street View images

Land-use maps are important references for urban planning and urban studies. Given the heterogeneity of urban land-use types, it is difficult to differentiate different land-use types based on overhead remotely sensed data. Google Street View (GSV) images, which capture the façades of building blocks along streets, could be better used to judge the land-use types of different building blocks based on their façade appearances. Recently developed scene classification algorithms in computer vision community make it possible to categorize different photos semantically based on various image feature descriptors and machine-learning algorithms. Therefore, in this study, we proposed a method to derive detailed land-use information at building block level based on scene classification algorithms and GSV images. Three image feature descriptors (i.e., scale-invariant feature transform-Fisher, histogram of oriented gradients, GIST) were used to represent GSV images of different buildings. Existing land-use maps were used to create training datasets to train support vector machine (SVM) classifiers for categorizing GSV images. The trained SVM classifiers were then applied to case study areas in New York City, Boston, and Houston, to predict the land-use information at building block level. Accuracy assessment results show that the proposed method is suitable for differentiating residential buildings and nonresidential buildings with an accuracy of 85% or so. Since the GSV images are publicly accessible, this proposed method would provide a new way for building block level land-use mapping in future.

[1]  John R. Jensen,et al.  Introductory Digital Image Processing: A Remote Sensing Perspective , 1986 .

[2]  Lorenzo Bruzzone,et al.  A technique for the selection of kernel-function parameters in RBF neural networks for classification of remote-sensing images , 1999, IEEE Trans. Geosci. Remote. Sens..

[3]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[4]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[5]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[6]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[7]  Fernando Bação,et al.  Self-organizing Maps as Substitutes for K-Means Clustering , 2005, International Conference on Computational Science.

[8]  Cordelia Schmid,et al.  Software - Histogram of oriented gradient object detection , 2006 .

[9]  Eli Shechtman,et al.  Matching Local Self-Similarities across Images and Videos , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  A. M. Hersperger,et al.  Containing urban sprawl—Evaluating effectiveness of urban growth boundaries set by the Swiss Land Use Plan , 2009 .

[11]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Yuji Murayama,et al.  A GIS Approach to Estimation of Building Population for Micro‐spatial Analysis , 2009, Trans. GIS.

[13]  Deren Li,et al.  Object Classification of Aerial Images With Bag-of-Visual Words , 2010, IEEE Geoscience and Remote Sensing Letters.

[14]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[15]  Krista A. Ehinger,et al.  SUN database: Large-scale scene recognition from abbey to zoo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Thomas Blaschke,et al.  Object based image analysis for remote sensing , 2010 .

[17]  Bryan C. Pijanowski,et al.  A backcast land use change model to generate past land use maps: application and validation at the Muskegon River watershed of Michigan, USA , 2010 .

[18]  Jie Shan,et al.  Building population mapping with aerial imagery and GIS data , 2011, Int. J. Appl. Earth Obs. Geoinformation.

[19]  Patricia Gober,et al.  Per-pixel vs. object-based classification of urban land cover extraction using high spatial resolution imagery , 2011, Remote Sensing of Environment.

[20]  Jungho Im,et al.  Support vector machines in remote sensing: A review , 2011 .

[21]  Victor Soto,et al.  Robust Land Use Characterization of Urban Landscapes using Cell Phone Data , 2011 .

[22]  R. C. Frohn,et al.  Segmentation and object-oriented classification of wetlands in a karst Florida landscape using multi-season Landsat-7 ETM+ imagery , 2011 .

[23]  Víctor Soto,et al.  Automated land use identification using cell-phone records , 2011, HotPlanet '11.

[24]  Dagmar Haase,et al.  Actors and factors in land-use simulation: The challenge of urban shrinkage , 2012, Environ. Model. Softw..

[25]  Shawn D. Newsam,et al.  Exploring Geotagged images for land-use classification , 2012, GeoMM '12.

[26]  Shawn D. Newsam,et al.  Geographic Image Retrieval Using Local Invariant Features , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Tao Yu,et al.  A hybrid method combining pixel-based and object-oriented methods and its application in Hungary using Chinese HJ-1 satellite images , 2013 .

[28]  César A. Hidalgo,et al.  The Collaborative Image of The City: Mapping the Inequality of Urban Perception , 2013, PloS one.

[29]  Timothy D. Meehan,et al.  Ecosystem-Service Tradeoffs Associated with Switching from Annual to Perennial Energy Crops in Riparian Zones of the US Midwest , 2013, PloS one.

[30]  Ramesh Raskar,et al.  Streetscore -- Predicting the Perceived Safety of One Million Streetscapes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[31]  M. Barthelemy,et al.  From mobile phone data to the spatial structure of cities , 2014, Scientific Reports.

[32]  Xiaomin Qiu,et al.  Incorporating road and parcel data for object-based classification of detailed urban land covers from NAIP images , 2014 .

[33]  Bryan C. Pijanowski,et al.  Comparing three global parametric and local non-parametric models to simulate land use change in diverse areas of the world , 2014, Environ. Model. Softw..

[34]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[35]  Vicente Ordonez,et al.  Learning High-Level Judgments of Urban Perception , 2014, ECCV.

[36]  Le Wang,et al.  An object-based SVM method incorporating optimal segmentation scale estimation using Bhattacharyya Distance for mapping salt cedar (Tamarisk spp.) with QuickBird imagery , 2015 .

[37]  Bryan C. Pijanowski,et al.  Land use legacies of the Ohio River Basin: Using a spatially explicit land use change model to assess past and future impacts on aquatic resources , 2015 .

[38]  Weixing Zhang,et al.  Assessing street-level urban greenery using Google Street View and a modified green view index , 2015 .

[39]  Chuanrong Zhang,et al.  Urban Land Use Information Retrieval Based on Scene Classification of Google Street View Images , 2016, SDW@GIScience.

[40]  Zhilei Lin,et al.  A support vector machine classifier based on a new kernel function model for hyperspectral data , 2016 .

[41]  Jie Shan,et al.  Object-based urban land cover classification using rule inheritance over very high-resolution multisensor and multitemporal data , 2016 .

[42]  Liangpei Zhang,et al.  The Fisher Kernel Coding Framework for High Spatial Resolution Scene Classification , 2016, Remote. Sens..

[43]  Yanfei Zhong,et al.  A spectral–structural bag-of-features scene classifier for very high spatial resolution remote sensing imagery , 2016 .

[44]  Peng Gong,et al.  Mapping Urban Land Use by Using Landsat Images and Open Social Data , 2016, Remote. Sens..

[45]  Qunying Huang,et al.  Channel bar feature extraction for a mining-contaminated river using high-spatial multispectral remote-sensing imagery , 2016 .

[46]  Chaogui Kang,et al.  Incorporating spatial interaction patterns in classifying and understanding urban land use , 2016, Int. J. Geogr. Inf. Sci..

[47]  Chi-Kuei Wang,et al.  Integration of full-waveform LiDAR and hyperspectral data to enhance tea and areca classification , 2016 .