Pipeline filters miss workers in rare low poses on construction sites
Auditing Quality Filters for Long-Tail Human Data Curation
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
When robots look for workers on construction sites, it’s important to spot people who are bending or kneeling, since these low poses are less common and harder to detect. The authors found that the automatic system’s quality checks tend to reject these rare low poses more often than normal standing ones. They tested this using different clues about the body’s shape and image features and confirmed the effect. This means that these low poses are not only rare but also difficult to find with current automatic detection methods.
What this means in practice
- •For construction robot developers: Improve worker safety systems by recognizing limitations of current detection filters in identifying hard-to-see low poses like kneeling or bending workers.
- •For video surveillance teams: Refine human detection pipelines to better handle rare body positions by analyzing how quality filters may bias training data against unusual poses.
Authors
Rishav Agarwal, Nirshal Chandra Sekar, Anirudh Vemula
Abstract
Robots on construction sites must detect workers who are kneeling or bending, which we call low poses. These workers can be lost from training datasets during automatic labeling. We study a pipeline that detects people, estimates their body joints using NLF, and groups similar poses. Low poses account for only about 2 percent of the retained examples. This low share may partly reflect the pipeline's quality filter, which rejects examples with low detection confidence or uncertain joint estimates. We examine this filtering using four alternative pose clues: bounding-box shape, vertical body span, pose grouping aligned to the scene's vertical direction, and image appearance. All four suggest that low poses are rejected by the filter more often. Separately, controlled simulated scenes show that a person detector fine-tuned on a public construction dataset misses more workers in these poses even when we correct their bounding box height is matched to that of standing workers. These findings suggest that low poses are scarce and hard to find, and we cannot rely on bounding boxes or poses for long-tail human data curation.