The COVID-19 pandemic highlighted the importance of understanding human behavior in cities to develop effective public health interventions. Crowd-sourced geospatial data available online can therefore serve as a valuable resource for analyzing such dynamics with Geospatial Artificial Intelligence (GeoAI). However, previous research often struggles with the complexity and heterogeneity of such data, and faces challenges in using multiple modalities and explicabilities. To address these challenges, the GeoEPI consortium introduced a novel GeoAI framework that integrates multimodal time series analysis of mobile phone data and geo-referenced social media posts to assess their relationship with COVID-19 incidence rates in Rio de Janeiro.
The study leverages a diverse set of data collected between April 2020 and August 2021 using a feature selection algorithm based on the previously used dependence measures by Chatterjee´s Xi. Crowd-sourced data has been used extensively to monitor human behavior and the environment. The data sets include mobile phone mobility data, geo-referenced social media posts (regarding their association with COVID-19 incidence rates using Chatterjee’s Xi measure of dependence in a feature selection context), COVID-19 case data and socio-demographic variables.
Research findings indicate a strong correlation between social media activity related to COVID-19, tourism, leisure activities, and infection rates. This correlation is supported by high dependence scores ranging up to 0.88. In contrast, mobility data showed low to medium dependence scores. The analysis revealed a correlation between socio-demographic factors, such as the proportion of the Indigenous population or social development, and the dependence scores of both mobility data and health- or leisure-related semantic topics on social media.
These findings underscore the value of integrating localized multimodal features into city-level epidemiological analysis and provide a method for effectively identifying them. In the broader context of GeoAI, the approach provides a framework for identifying and ranking relevant spatiotemporal features, enabling concrete insights before model building and increasing transparency when making predictions.
Reference: Hanny, D., Arifi, D., Knoblauch, S. et al. An explainable GeoAI approach for the multimodal analysis of urban human dynamics: a case study for the COVID-19 pandemic in Rio de Janeiro. Comput.Urban Sci. 5, 13 (2025). https://doi.org/10.1007/s43762-025-00172-2
