Urban Zoning Using Higher-Order Markov Random Fields on Multi-View Imagery Data

Urban zoning enables various applications in land use analysis and urban planning. As cities evolve, it is important to constantly update the zoning maps of cities to reflect urban pattern changes. This paper proposes a method for automatic urban zoning using higher-order Markov random fields (HO-MRF) built on multi-view imagery data including street-view photos and top-view satellite images. In the proposed HO-MRF, top-view satellite data is segmented via a multi-scale deep convolutional neural network (MS-CNN) and used in lower-order potentials. Street-view data with geo-tagged information is augmented in higher-order potentials. Various feature types for classifying street-view images were also investigated in our work. We evaluated the proposed method on a number of famous metropolises and provided in-depth analysis on technical issues.

[1]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[2]  Joachim Denzler,et al.  LAND COVER CLASSIFICATION OF SATELLITE IMAGES USING CONTEXTUAL INFORMATION , 2013 .

[3]  Richard Chbeir,et al.  Towards Better Land Cover Classification Using Geo-tagged Photographs , 2014, 2014 IEEE International Symposium on Multimedia.

[4]  Gordon Whitnall.  History of Zoning , 1931 .

[5]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[6]  Bolei Zhou,et al.  Recognizing City Identity via Attribute Analysis of Geo-tagged Images , 2014, ECCV.

[7]  Alexei A. Efros,et al.  What makes Paris look like Paris? , 2015, Commun. ACM.

[8]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[9]  Gustavo Camps-Valls,et al.  Structured output SVM for remote sensing image classification , 2009 .

[10]  Serge J. Belongie,et al.  Learning deep representations for ground-to-aerial geolocalization , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  B. Forster.  An examination of some problems and solutions in monitoring urban areas from satellite platforms , 1985 .

[12]  Mihai Datcu,et al.  Semantic Annotation of Satellite Images Using Latent Dirichlet Allocation , 2010, IEEE Geoscience and Remote Sensing Letters.

[13]  Brian P. Salmon,et al.  Multiview Deep Learning for Land-Use Classification , 2015, IEEE Geoscience and Remote Sensing Letters.

[14]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[15]  Jake Porway,et al.  A hierarchical and contextual model for aerial image understanding , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Sanja Fidler,et al.  HD Maps: Fine-Grained Road Segmentation by Parsing Ground and Aerial Images , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  A. Karnieli,et al.  Comparison of methods for land-use classification incorporating remote sensing and GIS inputs , 2011 .

[18]  S. Barr,et al.  INFERRING URBAN LAND USE FROM SATELLITE SENSOR IMAGES USING KERNEL-BASED SPATIAL RECLASSIFICATION , 1996 .

[19]  A. D. Gregorio,et al.  Land Cover Classification System (LCCS): Classification Concepts and User Manual , 2000 .

[20]  Philip J. Howarth,et al.  Land-use classification of SPOT HRV data using a cover-frequency method , 1992 .

[21]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[22]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[23]  Michele Volpi,et al.  Dense Semantic Labeling of Subdecimeter Resolution Images With Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[24]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[25]  Derek Nowrouzezahrai,et al.  Learning hatching for pen-and-ink illustration of surfaces , 2012, TOGS.

[26]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Piotr Tokarczyk,et al.  Features, Color Spaces, and Boosting: New Insights on Semantic Classification of Remote Sensing Images , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Scott Workman,et al.  A Unified Model for Near and Remote Sensing , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Michael A. Goldberg,et al.  Zoning: Its Costs and Relevance for the 1980s , 1980 .

[30]  Vibhav Vineet,et al.  Filter-Based Mean-Field Inference for Random Fields with Higher-Order Terms and Product Label-Spaces , 2012, International Journal of Computer Vision.

[31]  Michele Volpi,et al.  Semantic segmentation of urban scenes by learning local class interactions , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[32]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[33]  Henriette Cramer,et al.  Aesthetic capital: what makes london look beautiful, quiet, and happy? , 2014, CSCW.

[34]  Bolei Zhou,et al.  Learning Deep Features for Scene Recognition using Places Database , 2014, NIPS.

[35]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[36]  B. Pijanowski,et al.  Modeling the relationships between land use and land cover on private lands in the Upper Midwest, USA , 2000 .

[37]  Jake Porway,et al.  A Hierarchical and Contextual Model for Aerial Image Parsing , 2010, International Journal of Computer Vision.

[38]  Stefan Lee,et al.  Predicting Geo-informative Attributes in Large-Scale Image Collections Using Convolutional Neural Networks , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[39]  Vicente Ordonez,et al.  Learning High-Level Judgments of Urban Perception , 2014, ECCV.

[40]  Ramesh Raskar,et al.  Deep Learning the City: Quantifying Urban Perception at a Global Scale , 2016, ECCV.

[41]  Antonio Torralba,et al.  Unsupervised Non-parametric Geospatial Modeling from Ground Imagery , 2014, IEEE Winter Conference on Applications of Computer Vision.

[42]  Francesca Bovolo,et al.  A New Self-Training-Based Unsupervised Satellite Image Classification Technique Using Cluster Ensemble Strategy , 2015, IEEE Geoscience and Remote Sensing Letters.

[43]  Daniel P. Huttenlocher,et al.  Distance Transforms of Sampled Functions , 2012, Theory Comput..

[44]  Shawn D. Newsam,et al.  Proximate sensing: Inferring what-is-where from georeferenced photo collections , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[45]  Christian Heipke,et al.  AN ITERATIVE INFERENCE PROCEDURE APPLYING CONDITIONAL RANDOM FIELDS FOR SIMULTANEOUS CLASSIFICATION OF LAND COVER AND LAND USE , 2015 .

[46]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[47]  Shougeng Hu,et al.  Automated urban land-use classification with remote sensing , 2013 .