• Computer Science > Artificial Intelligence [Submitted on 18 Feb 2026] Title:GPSBench: Do Large Language Models Understand GPS Coordinates? • View PDF HTML (experimental)Abstract:Large Language Models (LLMs) are increasingly deployed in applications that interact with the physical world, such as navigation, robotics, or mapping, making robust geospatial reasoning a critical capability. • Despite that, LLMs’ ability to reason about GPS coordinates and real-world geography remains underexplored. • We introduce GPSBench, a dataset of 57,800 samples across 17 tasks for evaluating geospatial reasoning in LLMs, spanning geometric coordinate operations (e.g., distance and bearing computation) and reasoning that integrates coordinates with world knowledge. • Focusing on intrinsic model capabilities rather than tool use, we evaluate 14 state-of-the-art LLMs and find that GPS reasoning remains challenging, with substantial variation across tasks: models are generally more reliable at real-world geographic reasoning than at geometric computations. • Geographic knowledge degrades hierarchically, with strong country-level performance but weak city-level localization, while robustness to coordinate noise suggests genuine coordinate understanding rather than memorization.

Article Summaries:

  • Researchers have released GPSBench, a new benchmark comprising 57,800 samples across 17 geospatial tasks designed to probe large language models’ (LLMs) ability to reason about GPS coordinates. The tasks cover geometric operations such as distance and bearing calculations, as well as integration of coordinates with world knowledge. Evaluating 14 state‑of‑the‑art LLMs, the study finds that geospatial reasoning remains difficult, with models performing better on real‑world geographic queries than on precise geometric computations. Performance drops hierarchically from country to city level, yet models show robustness to coordinate noise, suggesting genuine understanding rather than memorization. The authors also demonstrate that augmenting training data with GPS coordinates can boost downstream geospatial performance, while fine‑tuning introduces trade‑offs between geometric accuracy and world knowledge. The dataset and code are publicly available.

Sources: