Ugo tracker improves multi-object video tracking with pixel masks

UGO: Unified Architecture for General Multi-Object Tracking by Segmentation

Computer Vision and Pattern Recognition

Summary

Tracking many objects in a video, especially when they change shape or appear close together, is hard. The authors designed UGO, a system that finds and follows all objects like the one you select in the first video frame. It uses a smart way to turn overlapping guesses into clear pixel-level outlines for each object. They also made a memory system that helps remember each object better across frames. This approach works better than past methods on several multi-object tracking tests.

What this means in practice

  • For video surveillance teams: Track and distinguish multiple moving objects from a single example with improved accuracy in crowded scenes.
  • For robotics developers: Enable robots to identify and follow various user-selected objects in cluttered or dynamic environments using pixel-level segmentation.

Authors

Jer Pelhan, Alan Lukezic, Matej Kristan

Abstract

General multi-object tracking (GMOT) tracks all instances of a user-specified category from a single first-frame exemplar. Prior work relies on bounding boxes and surrogate training, and struggles with non-rigid objects, crowded scenes, and distractors. We introduce UGO, a unified GMOT tracker that pairs a pretrained exemplar-conditioned detection head with an instance-propagation head in a common architecture. A novel training-free, energy-minimization consolidation method converts overlapping proposals into exclusive pixel-wise masks and detections, resolving over-segmentation, duplicates, and conflicts. A hierarchical memory spanning global and instance levels improves recall and per-instance segmentation accuracy using a new memory management protocol. UGO sets a new state-of-the-art on GMOT benchmarks and video object counting, and is competitive with specialist MOT methods, establishing a strong paradigm for unified, open-category multi-object tracking.