Ai method targets expert review to improve biodiversity surveys
Targeted Review for AI-Assisted Biodiversity Surveys: Active Continuous-Score Occupancy Modeling
Machine LearningComputational Engineering, Finance, and Science
Summary
Scientists use machine learning to label pictures and sounds of wildlife, but these computer guesses can be wrong and cause mistakes in understanding nature. The authors created a new method called ACORN that helps decide which machine-labeled data experts should check carefully to save time while still getting good results. Their method picks the most useful samples for review based on how they affect the final ecological analysis, not just to improve the computer’s accuracy. Tests showed ACORN can get conclusions close to fully checked data with much less expert work.
What this means in practice
- •For ecological monitoring teams: Optimize expert effort in reviewing machine-labeled wildlife data to improve accuracy in species occurrence estimates with limited review budgets.
- •For environmental consulting firms: Reduce costs and improve reliability of ecological surveys by selectively validating AI-generated species presence data with expert reviews.
Authors
Timm Haucke, Lauren Harrell, Justin Kay, Mary Clapp, Sara Beery
Abstract
We increasingly use machine learning to label scientific datasets. The models we develop and deploy are improving all the time, but they are not and will likely never be perfect. Mistakes matter, as errors can propagate into our scientific understanding, particularly when systematically biased. Very reasonably, scientists thus review substantial proportions of ML-generated labels to verify or correct mistakes in pursuit of ensuring their scientific findings are not biased by ML. In this work, we focus on helping scientists optimally allocate this reviewing effort relative to their scientific goals. We focus on a specific class of scientists (ecologists) and a specific, widespread, and impactful modeling target (occupancy modeling, which estimates where species are likely to occur, conditioned on environmental factors). We introduce Active Continuous-Score Occupancy Modeling (ACORN), a method that incorporates ML predictions into occupancy models and strategically selects samples for expert review that are maximally informative for downstream ecological analysis. Across camera-trap and bioacoustic datasets, our method recovers ecological conclusions close to those obtained from fully human-labeled data, while requiring substantially fewer expert reviews than non-targeted review policies. Our results suggest that ML-assisted scientific workflows should optimize expert effort for downstream inference, rather than for classifier accuracy alone, especially when human review budget is limited. Our code is available at https://github.com/timmh/acorn