Sampling-aware benchmark reveals strengths of species distribution models
SAGE: A sampling-aware global evaluation benchmark for species distribution modeling
Machine Learning
Summary
Knowing where different plant species live helps protect nature. The authors created a new way to test how well computer models predict plant locations worldwide, by carefully considering how often and how well plants have been observed. They found that some advanced models work better for rare plants, especially when they correct for biased data. Their benchmark helps to understand when certain models can be trusted and supports improving conservation tools.
What this means in practice
- •For conservation planners: Select reliable species distribution models for rare plants by using the benchmark to assess model trustworthiness in low-data situations.
- •For ecological data engineers: Incorporate bias-correction techniques from the benchmark into deep learning systems to improve species occurrence predictions.
Authors
Emilia Arens, Nina van Tiel, Robin Zbinden, Damien Robert, Lukas Drees, Chiara Vanalli, Benjamin Kellenberger, Niklaus E. Zimmermann, Loïc Pellissier, Devis Tuia, Jan Dirk Wegner
Abstract
Knowing where species occur is fundamental for biodiversity research and conservation. Species distribution models (SDMs) link species observations to environmental conditions to estimate their spatial distribution. However, accuracy varies with the underlying data and models, making it essential to know for which species models can be trusted. Deep-learning-based SDMs ("DeepSDMs") now jointly model thousands of species, drawing on hundreds of millions of community-science records. At this scale, averaging performance hides substantial species-level variability, particularly for rare species, often of greatest conservation concern. Records are also strongly biased, making occurrence counts misleading. Accounting for these factors is essential for a reliable and informative evaluation of multi-species SDMs. Here, we introduce a Sampling-Aware Global Evaluation (SAGE) benchmark, combining GBIF records for training with sPlotOpen vegetation plots for presence-absence evaluation across 5771 plant species. We propose an evaluation framework that groups species based on two properties, sampling effort and relative prevalence, which describe how densely a species' range is sampled and how frequently the species is recorded. Evaluating single-species SDMs and multi-species DeepSDMs, we find that Random Forests and DeepSDMs perform best overall, but neither dominates: DeepSDMs outperform single-species SDMs for infrequently recorded species while offering no consistent advantage for well-sampled ones. Crucially, this advantage emerges only when established bias-correction practices, such as spatial thinning and reweighting, are carried over to the deep-learning setting. SAGE helps identify the species and data conditions for which a given approach is beneficial, thereby supporting the development of more transparent and ecologically credible SDMs. Data and code: https://earens.github.io/sage/