Pedestrian route safer estimates from image captions match direct image analysis
Caption-Mediated Perceived-Safety Estimation for Pedestrian Routing
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
This research looks at how to estimate how safe pedestrians feel when walking by analyzing street photos with computer descriptions. Instead of only using raw image data, the researchers use generated text captions to explain what the image shows, making the safety scoring easier to understand. They tested this approach on over 654,000 images and compared it to direct image analysis, finding similar performance. Real-world tests showed modest but meaningful agreement with human safety ratings. They also found that for longer walking trips, people tend to pick slightly safer routes even if it means a small detour.
What this means in practice
- •For urban planners: Use caption-based safety scores from street images to evaluate neighborhood walkability and plan safer pedestrian routes.
- •For mobile app developers: Implement transparent safety scoring in walking directions by leveraging text captions from street-level photos for user inspection.$Commercial implications: It enables safer route recommendation features in navigation apps that offer explainable safety assessments based on accessible descriptions.
Authors
Simon Parkinson, Paloma Liu, Wei Zheng, Mohammadreza Sheikhfathollahi
Abstract
This paper presents an explainable approach to pedestrian routing, in which perceived safety is estimated from street-level imagery through an explicit natural-language intermediate representation. A vision--language model caption is generated and stored before any scoring is undertaken, and the perceived-risk class is derived entirely from structured features of that stored text, so that every segment score remains inspectable by the user. Nine captioning conditions across five model families are benchmarked against a direct Contrastive Language--Image Pre-training (CLIP) image-embedding baseline under an identical downstream pipeline, and the caption-mediated representation is found to reach parity with the image embedding rather than to trail it. The approach was deployed over 654,115 images covering 36 electoral wards in two locations in Northern England (Manchester and Huddersfield). Independent field validation against 3,669 locally collected ratings of 494 images across 70 participant sessions established agreement that is statistically significant but modest, at $r=0.262$, against a measured noise ceiling of 0.737 imposed by disagreement between raters. A single-use confirmatory test then found that a pipeline 44\% stronger on the supervised benchmark did not produce measurable improvement in the field ($r=0.250$, $p=0.84$), so the benchmark gains did not predict the deployment gains in this case. Routing behaviour varies systematically with journey length. There is negligible change below 1\,km, reaching a median increase of 12.78\% in low-risk route length for a median detour of 2.73\% on journeys of 3 to 6 km.