Hot weather not only impacts upon human physical comfort and health, but also impacts the way that people access and experience active travel options such as walking and cycling. By evaluating the street thermal environment of a city alongside an assessment of those communities that are the most vulnerable to the effects of heat, we can prioritise areas in which heat mitigation interventions are most needed. In this paper, we propose a new approach for policy makers to determine where to delegate limited resources for heat mitigation with most effective outcomes for the communities. We use eye-level street panorama images and community profiles to provide a bottom-up, human-centred perspective of the city scale assessment, highlighting the situation of urban tree shade provision throughout the streets in comparison with environmental and social-economic status. The approach leverages multiple sources of spatial data including satellite thermal images, Google street view (GSV) images, land use and demographic census data. A deep learning model was developed to automate the classification of streetscape types and percentages at the street- and eye-view level. The methodology is metrics based and scalable which provides a data driven assessment of heat-related vulnerability. The findings of this study first contribute to sustainable development by developing a method to identify geographical areas or neighbourhoods that require heat mitigation; and enforce policies improving tree shade on routes, as a heat adaptation strategy, which will lead to increasing active travel and produce significant health benefits for residents. The approach can be also used to guide post COVID-19 city planning and design.