Version 1
: Received: 17 September 2024 / Approved: 17 September 2024 / Online: 17 September 2024 (12:34:44 CEST)
How to cite:
de Moraes Vestena, K.; Phillipi Camboim, S.; Brovelli, M. A.; Rodrigues dos Santos, D. Investigating the Performance of Open Vocabulary Classification Algorithms for Pathway and Surface Material Detection in Urban Environments. Preprints2024, 2024091321. https://doi.org/10.20944/preprints202409.1321.v1
de Moraes Vestena, K.; Phillipi Camboim, S.; Brovelli, M. A.; Rodrigues dos Santos, D. Investigating the Performance of Open Vocabulary Classification Algorithms for Pathway and Surface Material Detection in Urban Environments. Preprints 2024, 2024091321. https://doi.org/10.20944/preprints202409.1321.v1
de Moraes Vestena, K.; Phillipi Camboim, S.; Brovelli, M. A.; Rodrigues dos Santos, D. Investigating the Performance of Open Vocabulary Classification Algorithms for Pathway and Surface Material Detection in Urban Environments. Preprints2024, 2024091321. https://doi.org/10.20944/preprints202409.1321.v1
APA Style
de Moraes Vestena, K., Phillipi Camboim, S., Brovelli, M. A., & Rodrigues dos Santos, D. (2024). Investigating the Performance of Open Vocabulary Classification Algorithms for Pathway and Surface Material Detection in Urban Environments. Preprints. https://doi.org/10.20944/preprints202409.1321.v1
Chicago/Turabian Style
de Moraes Vestena, K., Maria Antonia Brovelli and Daniel Rodrigues dos Santos. 2024 "Investigating the Performance of Open Vocabulary Classification Algorithms for Pathway and Surface Material Detection in Urban Environments" Preprints. https://doi.org/10.20944/preprints202409.1321.v1
Abstract
Mapping pavement types, especially in sidewalks, is essential for urban planning and
mobility studies. Identifying pavement materials is a key factor in assessing mobility, such as
walkability and wheelchair usability. However, satellite imagery in this scenario is limited, and
in-situ mapping can be costly. A promising solution is to extract such geospatial features from
street-level imagery. This study explores using open vocabulary classification algorithms to
segment and identify pavement types and surface materials in this scenario. Our approach uses
Large Language Models (LLMs) to improve the accuracy of classifying different pavement types.
The methodology involves two experiments: the first uses free prompting with random
street-view images, employing Grounding Dino and SAM algorithms to assess performance across
categories. The second experiment evaluates standardized pavement classification using the
Deep-Pavements Dataset and a fine-tuned CLIP algorithm optimized for detecting
OSM-compliant pavement categories. The study presents open resources, such as the Deep
Pavements Dataset and a fine-tuned CLIP-based model, demonstrating a significant improvement
in the True Positive Rate (TPR) from 56.04% to 93.5%. Our findings highlight both the potential
and limitations of current open vocabulary algorithms and emphasize the importance of diverse
training datasets. This study advances urban feature mapping by offering a more intuitive and
accurate approach to geospatial data extraction, enhancing urban accessibility and mobility
mapping.
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
,
