Summary
Crowd-counting models, which estimate the number of people in images, can lose accuracy when images are unclear or when these models are simplified to run on small devices. The authors created a testing setup called CrowdCC that combines different real-world image errors with various levels of model simplification to see how well these models work together. They found that some kinds of image problems hurt performance the most, and that making models simpler (quantizing) changes their accuracy in unpredictable ways. This means you can’t just look at clean tests or model size to know how well a model will work in practice. The authors also provide code for others to use this testing method.
What this means in practice
- •For edge device developers: Measure real-world performance trade-offs of crowd counting models under image corruption and quantization to optimize deployment on limited hardware.
- •For surveillance system engineers: Select crowd counting models that maintain accuracy despite degraded video quality and device constraints for reliable monitoring under imperfect conditions.
Abstract
Crowd-counting models deployed on edge devices face two shifts: inputs are degraded along the capture and transmission pipeline, while models are quantized to satisfy resource constraints. Yet corruption robustness and deployment quantization are typically evaluated separately. We introduce CrowdCC, a controlled corruption benchmark that jointly evaluates these effects using 23 deployment-motivated corruptions at three severity levels, four crowd-counting datasets, and 26 models using A100 TensorRT engines at FP32, FP16, and post-training INT8 precision, paired with throughput and energy measurements from corresponding Jetson Orin NX configurations. The corruptions span six families, with head annotations adjusted when geometry or visibility changes. Our evaluation shows that capture-chain corruptions consistently produce the largest degradation across the evaluated models, while crowd density affects absolute and relative error differently. Clean accuracy does not fully predict corrupted performance. INT8 alters corruption robustness substantially more than FP16, with no systematic direction, and does not consistently improve latency or energy. INT8 inference efficiency is strongly associated with reformat overhead in the compiled engine, while slowdowns are also observed under limited INT8 coverage. Precision changes corrupted-error rankings and robustness-cost frontiers, showing why model selection must consider corruption robustness and measured deployment cost together. The code is available at https://github.com/Tulane-Computer-Architecture-Systems/CrowdCC-Release.