What role does the human-in-the-loop play in AI-assisted mapping?
AI-assisted mapping is rapidly changing how geospatial data is created, enabling features to be generated at a scale and speed that was previously unimaginable. Rather than replacing mappers, these workflows increasingly rely on humans to validate automatically generated data.
Experiments by the Humanitarian OpenStreetMap Team (HOT) have shown that AI assistance can amplify the efforts of mappers, mainly by accelerating digitization. However, this speed does not automatically translate into higher-quality data. Instead, much of the improvement happens during human validation, where mappers refine and add detail.
Traditionally, volunteered geographic information (VGI) has relied on the “wisdom of the crowd” for quality assurance, with multiple contributors identifying and correcting errors over time. The introduction of state-of-the-art GeoAI models has accelerated this process, enabling mapping at a scale and speed previously unimaginable. Despite these technical advances, wider adoption still lags, largely due to concerns around accuracy and trust.
Yet, AI-assisted mapping is already reshaping how we approach mapping. Now, editors need to understand how AI-generated features behave, identify recurring patterns in their strengths and weaknesses, and guide improvements over time.
In OpenStreetMap (OSM), these shifts are actively debated. Community members expressed “mixed sentiments” about integrating AI roads, raising concerns over trust, authenticity, and data quality. While these concerns may be valid, the nature of interaction with AI-generated data remains a black box. To what extent does their actions improve, degrade or do human simply approve the AI-generated features out of trust? When that happens, errors can slip through or be introduced when the algorithm gets things wrong.
This lack of empirical understanding contributes to ambiguity around the human role in the loop. It also fuels broader questions around responsibility, trust, and data quality in AI-assisted mapping pipelines, ultimately slowing adoption. What we know is that, when a mapper adjusts a node, it is because they have identified something they want to correct.
To better understand these actions, we designed a controlled experiment to measure how humans modify AI-generated road geometries at the atomic level. Participating were Youth Mappers of two universities in Ghana, University of Ghana Youth Mappers (UGYM) and University of Cape Coast Youth Mappers (UCCYM). A total to 37 people participated in the experiment with 13 people from UCCYM and 24 people from UGYM. Each mapped a grid size of 0.8 X 0.8 degree approximating to ~7,885km sq, focusing on regions they were familiar with.
AI-assisted mapping with the University of Ghana Youth Mappers
We used two approaches: independent and cross redundancy mapping. In the independent method, each participant validated and mapped a unique grid. In the redundancy method, same grid was assigned to multiple participants without their knowledge, allowing us to compare how different individuals edited the same AI-generated data.
Using data provided by meta for its AI-assisted Mapping, participants edit features in a controlled JOSM environment, enabling direct comparison between AI-only and human-edited geometries. They were given total freedom and instructed to do anything they deem right to make the data correct in their sight and they are free to accept data if they are certain there is nothing more they can add.
Additionally, participants provided feedback on their perceived AI quality, efforts required to edit, their trust for the AI-data, the nature of edits their grid required, and confidence in their final outputs.
This combination of behavioral data and self-reported perception gives us a richer understanding of how humans engage with AI-generated map features. It is an exciting step towards answering these questions:
- What exactly does the human in the loop do?
- What does these actions tells us about the quality of the AI data?
- How can we translate these human actions into quality indicators to improve the current validation workflow?
Looking ahead
This work is a first step toward making the role of the human in AI-assisted mapping more visible and measurable. While human-in-the-loop workflows are often treated as a guarantee of quality, we still lack a clear understanding of what mappers actually contribute, and how their actions shape the final data.
Building on these insights, we will focus on breaking down mapping behavior into fundamental editing actions and examining how these relate to both data quality and mapper perception. In parallel, we will explore whether trust in AI influences how critically mappers validate the data.
The main outcome will be the development of indicators that help interpret AI-generated data during validation. What do these human edits signal about the underlying quality of the data? And how can these signals be used to guide mapping workflows?
For the OpenStreetMap community, as well as anyone working with the OSM data, this research has immediate relevance. As the volume of AI-generated data continues to grow, validators are increasingly faced with more data than they can realistically review. Better indicators could help prioritize effort and assess whether data is fit for purpose.
More broadly, this work contributes to an ongoing effort to bring greater transparency to the interaction between humans and AI in open mapping. Understanding how humans shape the data, will be key to building trust and sustaining the future of volunteered geographic information in an AI-assisted mapping.
Reference:
Andorful, F., Herfort, B., Melanda, E. A., Antonio, N. D., Zipf, A., & Camboim, S. P. (2025). Transparency and Trust in Collaborative Mapping: Concerns and Dilemmas in AI-Assisted Road Integration within OpenStreetMap. Annals of the American Association of Geographers, 1–22. Transparency and Trust in Collaborative Mapping: Concerns and Dilemmas in AI-Assisted Road Integration within OpenStreetMap
Memduhoğlu, A. (2025). Towards AI-Assisted Mapmaking: Assessing the Capabilities of GPT-4o in Cartographic Design. ISPRS International Journal of Geo-Information, 14(1), 35. https://doi.org/10.3390/ijgi14010035
Title image was created with AI-generated road data from Microsoft (2020).
