Towards Global-Scale Crowd+AI Techniques to Map and Assess Sidewalks for People with Disabilities

There is a lack of data on the location, condition, and accessibility of sidewalks across the world, which not only impacts where and how people travel but also fundamentally limits interactive mapping tools and urban analytics. In this paper, we describe initial work in semi-automatically building a sidewalk network topology from satellite imagery using hierarchical multi-scale attention models, inferring surface materials from street-level images using active learning-based semantic segmentation, and assessing sidewalk condition and accessibility features using Crowd+AI. We close with a call to create a database of labeled satellite and streetscape scenes for sidewalks and sidewalk accessibility issues along with standardized benchmarks.

[1]  Andres Sevtsuk,et al.  Mapping the walk: A scalable computer vision approach for generating sidewalk network datasets from aerial imagery , 2023, Comput. Environ. Urban Syst..

[2]  Jon E. Froehlich,et al.  Visualizing Urban Accessibility: Investigating Multi-Stakeholder Perspectives through a Map-based Design Probe Study , 2022, CHI.

[3]  Jianzhe Lin,et al.  CitySurfaces: City-Scale Semantic Segmentation of Sidewalk Materials , 2022, Sustainable Cities and Society.

[4]  Roberto M. Cesar,et al.  Sidewalk Measurements from Satellite Images: Preliminary Findings , 2021, ArXiv.

[5]  Amy K. Lobben,et al.  Squeaky wheels: Missing data, disability, and power in the smart city , 2021, Big Data Soc..

[6]  R. Gould,et al.  Are communities in the United States planning for pedestrians with disabilities? Findings from a systematic evaluation of local government barrier removal plans , 2020 .

[7]  Karan Sapra,et al.  Hierarchical Multi-Scale Attention for Semantic Segmentation , 2020, ArXiv.

[8]  Graham Dove,et al.  Urban Mosaic: Visual Exploration of Streetscapes Using Large-Scale Image Data , 2020, CHI.

[9]  Jon E. Froehlich,et al.  Deep Learning for Automatically Detecting Sidewalk Accessibility Problems Using Streetscape Imagery , 2019, ASSETS.

[10]  Yang Zhao,et al.  Deep High-Resolution Representation Learning for Visual Recognition , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Adam Van Etten,et al.  City-Scale Road Extraction from Satellite Imagery v2: Road Speeds and Travel Times , 2019, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[12]  Kai Zhang,et al.  Road Network Extraction from Satellite Images Using CNN Based Segmentation and Tracing , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[13]  Jon Froehlich,et al.  Project Sidewalk: A Web-based Crowdsourcing Tool for Collecting Sidewalk Accessibility Data At Scale , 2019, CHI.

[14]  Johannes Schöning,et al.  Grand challenges in accessible maps , 2019, Interactions.

[15]  Weijia Li,et al.  Semantic Segmentation-Based Building Footprint Extraction Using Very High-Resolution Satellite Images and Multi-Source GIS Data , 2019, Remote. Sens..

[16]  Shawn D. Newsam,et al.  Improving Semantic Segmentation via Video Propagation and Label Relaxation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Jon Froehlich,et al.  Interactively Modeling and Visualizing Neighborhood Accessibility at Scale: An Initial Study of Washington DC , 2018, ASSETS.

[18]  David J. DeWitt,et al.  RoadTracer: Automatic Extraction of Road Networks from Aerial Images , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Mahdi Aghaabbasi,et al.  Evaluating the capability of walkability audit tools for assessing sidewalks , 2018 .

[20]  Peter Kontschieder,et al.  The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[21]  Bolei Zhou,et al.  Scene Parsing through ADE20K Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Vladimir Iglovikov,et al.  Satellite Imagery Feature Detection using Deep Convolutional Neural Network: A Kaggle Competition , 2017, ArXiv.

[23]  Vijay Vasudevan,et al.  Does the built environment moderate the relationship between having a disability and lower levels of physical activity? A systematic review. , 2017, Preventive medicine.

[24]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Yi Yang,et al.  Attention to Scale: Scale-Aware Semantic Image Segmentation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Mani Golparvar-Fard,et al.  Detection, classification, and mapping of U.S. traffic signs using google street view images for roadway inventory management , 2015 .

[27]  Marie Boltz,et al.  The Neighborhood Environment: Perceived Fall Risk, Resources, and Strategies for Fall Prevention. , 2015, The Gerontologist.

[28]  Wei Liu,et al.  ParseNet: Looking Wider to See Better , 2015, ArXiv.

[29]  Jon E. Froehlich,et al.  Improving Public Transit Accessibility for Blind Riders by Crowdsourcing Bus Stop Landmark Locations with Google Street View: An Extended Analysis , 2015, ACM Trans. Access. Comput..

[30]  J. Sanford,et al.  Physical Environmental Barriers to Community Mobility in Older and Younger Wheelchair Users , 2015 .

[31]  Jon Froehlich,et al.  Tohme: detecting curb ramps in google street view using crowdsourcing, computer vision, and machine learning , 2014, UIST.

[32]  Jon Froehlich,et al.  An Initial Study of Automatic Curb Ramp Detection with Crowdsourced Verification Using Google Street View Images , 2013, HCOMP.

[33]  Jon Froehlich,et al.  Improving public transit accessibility for blind riders by crowdsourcing bus stop landmark locations with Google street view , 2013, ASSETS.

[34]  Jon Froehlich,et al.  Combining crowdsourcing and google street view to identify street-level accessibility problems , 2013, CHI.

[35]  Jon Froehlich,et al.  A feasibility study of crowdsourcing and google street view to determine sidewalk accessibility , 2012, ASSETS '12.

[36]  Keith M. Christensen,et al.  Effects of Perceived Neighborhood Characteristics and Use of Community Facilities on Physical Activity of Adults With and Without Disabilities , 2010, Preventing chronic disease.

[37]  Kesheng Wu,et al.  Fast connected-component labeling , 2009, Pattern Recognit..

[38]  Laura A Talbot,et al.  Falls in young, middle-aged and older community dwelling adults: perceived cause, environmental factors and injury , 2005, BMC public health.

[39]  Azriel Rosenfeld,et al.  Sequential Operations in Digital Picture Processing , 1966, JACM.

[40]  Jon E. Froehlich,et al.  Exploring Early Solutions for Automatically Identifying Inaccessible Sidewalks in the Physical World using Google Street View , 2014 .

[41]  Michael Christopher,et al.  Pedestrian Mobility and Safety: A Key to Independence for Older People , 2006 .