Under the main theme “Spatial data for design” the 26th AGILE conference will be held this week in Delft, Netherlands. Besides a variety of geo-spatial related themes, disaster mapping still plays a crucial role in GIScience. Simon Groß and Sven Lautenbach will present the paper “Exploring MapSwipe as a Crowdsourcing Tool for (Rapid) Damage Assessment: The Case of the 2021 Haiti Earthquake” that explores a potential new use case for the app MapSwipe.
Fast and reliable geographic information is vital in disaster management. In the late 2000s, crowdsourcing emerged as a powerful method to provide this information. Base mapping through crowdsourcing is already well established in relief workflows. However, crowdsourced post-disaster damage assessment is researched but not yet institutionalized. Therefore, the paper written by members of HeiGIT focuses on the developement of a damage assessment approach based on MapSwipe, an established mobile application for crowdsourced base mapping. The approach was tested for a case study after the 2021 Haiti earthquake.
A New Use Case for MapSwipe?
Up until now, MapSwipe is mainly used to collect data for basemapping, where users can tag tiles if they spot for example a building or a street.
In order to collect crowdsourced damage assessment data, a new project type was introduced to MapSwipe. Users can now tag “damaged”, “maybe damaged” or “bad imagery” (no tag means no damage), while seeing a before and after aerial picture.
To this day, there is no institutionalized platform for crowdsourced damage assessment. Therefore, this project analyses the potential of MapSwipe to fill this gap.
The Paper
In the paper “Exploring MapSwipe as a Crowdsourcing Tool for (Rapid) Damage Assessment: The Case of the 2021 Haiti Earthquake” the quality of MapSwipe data is analyzed by comparing it to a reference dataset, created by the authors. This is framed by the following research question:
RQ1: How well does MapSwipe perform as a tool for crowdsourced building damage classification?
Then, the MapSwipe data is compared to an established (but not crowdsourced) damage assessment workflow by the Copernicus Emergency Management Service (CEMS):
RQ2: How does crowdsourced building damage information from MapSwipe compare to institutionalized information obtained through CEMS?
The Study Area
On the 14th of August 2021 a 7.2 magnitude earthquake hit the southwestern peninsula of Haiti (Figure 2). The specific study area was the city of Les Cayes 40 km southwest of the epicenter, which was overlayed with MapSwipe tiles (Figure 3).
Methodology
The study design is straight forward. For each tile, there is six to eight answers by volunteers. Different thresholds were then explored to classify a tile as damaged, e.g. 30 % of users tagged “damage” or 60 % of users tagged “damage”.
These different threshold were then used to perform a quality assessment with the reference dataset for RQ1 and an agreement analysis using Cohen’s Kappa for RQ2.
Results
RQ1
The results give a positive outlook for the new use case. F1 peaks at a threshold of 35 % suggesting that a lower threshold than simple majority voting can get a better F1 score.
RQ2
For RQ2 the Cohen’s Kappa values suggest only slight agreement with copernicus data, peaking at 0.16 with a much stricter threshold for a MapSwipe damage definition of 65 %.
Discussion
Generally the F1 score of > 0.6 gives a positive outlook for damage detection with MapSwipe. The damage definition threshold could also be adjusted depending on the use case, e.g. using a higher threshold for first response and a lower one for more detailed and in depth relief efforts later. Combining crowdsourcing with established methods like CEMS could also be interesting for future research, as well as the potential of combining crowdsourcing with automation and deep learning approaches.
Also new functionalities could be included in MapSwipe, for example using point placing for tagging damaged buildings instead of tagging the whole tile and push notifications to communicate urgency in case of a disaster.
Generally, incorporating damage assessment into MapSwipe could also help to catch new users through the media attention a disaster creates, which could help creating a larger community for the app.
The paper will be presented on the AGILE conference on the 15.06.2023.
The codes for the paper are available on this GitHub repository.
For full paper see:
Groß, S., Herfort, B., Marx, S., and Zipf, A.: Exploring MapSwipe as a Crowdsourcing Tool for (Rapid) Damage Assessment: The Case of the 2021 Haiti Earthquake, AGILE GIScience Ser., 4, 5, https://doi.org/10.5194/agile-giss-4-5-2023, 2023.