Street-Frontage-Net: urban image classification using deep convolutional neural networks

ABSTRACT Quantifying aspects of urban design on a massive scale is crucial to help develop a deeper understanding of urban designs elements that contribute to the success of a public space. In this study, we further develop the Street-Frontage-Net (SFN), a convolutional neural network (CNN) that can successfully evaluate the quality of street frontage as either being active (frontage containing windows and doors) or blank (frontage containing walls, fences and garages). Small-scale studies have indicated that the more active the frontage, the livelier and safer a street feels. However, collecting the city-level data necessary to evaluate street frontage quality is costly. The SFN model uses a deep CNN to classify the frontage of a street. This study expands on the previous research via five experiments. We find robust results in classifying frontage quality for an out-of-sample test set that achieves an accuracy of up to 92.0%. We also find active frontages in a neighbourhood has a significant link with increased house prices. Lastly, we find that active frontage is associated with more scenicness compared to blank frontage. While further research is needed, the results indicate the great potential for using deep learning methods in geographic information extraction and urban design.

[1]  P. Moran The Interpretation of Statistical Maps , 1948 .

[2]  P. Moran,et al.  Rank correlation and product-moment correlation. , 1948, Biometrika.

[3]  J. Jacobs The Death and Life of Great American Cities , 1962 .

[4]  John A. Henning,et al.  The Determinants of Residential Property Values with Special Reference to Air Pollution , 1967 .

[5]  S. Rosen Hedonic Prices and Implicit Markets: Product Differentiation in Pure Competition , 1974, Journal of Political Economy.

[6]  Murray Silverstein,et al.  A Pattern Language , 1977 .

[7]  Ronald Wiedenhaft Cities for People , 1981 .

[8]  Raymond B. Palmquist Estimating the Demand for the Characteristics of Housing , 1984 .

[9]  L. Anselin Spatial Econometrics: Methods and Models , 1988 .

[10]  Paul Cheshire,et al.  On the Price of Land and the Value of Amenities , 1995 .

[11]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[12]  Jan Gehl,et al.  Life Between Buildings: Using Public Space , 2003 .

[13]  R. Huxford Safer places ? the planning system and crime prevention , 2004 .

[14]  J. Jacobs,et al.  Living Room: Rematerialising Home , 2008 .

[15]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[16]  Alexei A. Efros,et al.  What makes Paris look like Paris? , 2015, Commun. ACM.

[17]  Jim Berry,et al.  Hedonic modelling of high street retail properties: a quality design perspective , 2013 .

[18]  Ramesh Raskar,et al.  Streetscore -- Predicting the Perceived Safety of One Million Streetscapes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[19]  Wei Pan,et al.  The relationship between the quality of active frontages and public perceptions of public spaces , 2013, URBAN DESIGN International.

[20]  Alexei A. Efros,et al.  City Forensics: Using Visual Elements to Predict Non-Visual City Attributes , 2014, IEEE Transactions on Visualization and Computer Graphics.

[21]  Bolei Zhou,et al.  Recognizing City Identity via Attribute Analysis of Geo-tagged Images , 2014, ECCV.

[22]  Bolei Zhou,et al.  Learning Deep Features for Scene Recognition using Places Database , 2014, NIPS.

[23]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[24]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[25]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[27]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[28]  Conrad Kickert,et al.  Active centers – interactive edges: The rise and fall of ground floor frontages , 2016 .

[29]  Ramprasaath R. Selvaraju,et al.  Grad-CAM: Why did you say that? Visual Explanations from Deep Networks via Gradient-based Localization , 2016 .

[30]  Hui Wang,et al.  A machine learning-based method for the large-scale evaluation of the qualities of the urban environment , 2017, Comput. Environ. Urban Syst..

[31]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Tobias Preis,et al.  Using deep learning to quantify the beauty of outdoor places , 2017, Royal Society Open Science.

[33]  Yao Shen,et al.  An application of convolutional neural network in street image classification: the case study of london , 2017, GeoAI@SIGSPATIAL.

[34]  Jonathan Krause,et al.  Using deep learning and Google Street View to estimate the demographic makeup of neighborhoods across the United States , 2017, Proceedings of the National Academy of Sciences.

[35]  Quanshi Zhang,et al.  Visual interpretability for deep learning: a survey , 2018, Frontiers of Information Technology & Electronic Engineering.

[36]  Paolo Santi,et al.  Investigating the association between streetscapes and human walking activities using Google Street View and human trajectory data , 2018, Trans. GIS.

[37]  Heeket Mehta,et al.  Google Maps , 2019, International Journal of Computer Applications.

[38]  S. Law,et al.  Take a Look Around: Using Street View and Satellite Images to Estimate House Prices , 2019 .