Video tracking method improves identity summaries with diverse features
Identity-Centric Video Summarization via Hierarchical Fusion of Biometric, Appearance, and 3D Body Features
Computer Vision and Pattern Recognition
Summary
Watching videos with many people can be confusing, so it's helpful to create short summaries of each person’s appearance and actions. The authors made a new way to track people in videos by combining facial recognition, body shapes, and looks. Their method keeps better track even when faces are blocked or the video is blurry. They then make short video summaries that capture key moments for each individual, showing more variety and useful information than older methods.
What this means in practice
- •For security system designers: Create clearer identity-based video highlights to speed up reviewing footage with multiple people in crowded scenes.
- •For video content editors: Generate diverse and informative curated clips of individuals automatically from long videos with better temporal coverage and clarity.
Tested on one dataset.
Authors
Milad Mirjalili, Enrique Alegre Gutiérrez, Eduardo Fidalgo Fernández, Víctor González Castro, Rocío Alaiz Rodríguez, Manuel Castejón Limas
Abstract
This work presents a video summarization algorithm based on multi-object tracking and person reidentification. We integrate facial embeddings, 3D body-shape features, and visual appearance into a unified tracking framework. These representations enable hierarchical identity assignment and tracking through bidirectional anchoring, which robustly recovers trajectories under severe occlusion or low visual quality. From these stable trajectories, we generate a compact set of summaries for each identity. We select keyframes using a multi-factor weighting scheme that optimizes biometric clarity, social interaction, and motion dynamics, while Adaptive Non-Maximum Suppression ensures temporal diversity. Evaluation on a custom dataset demonstrates tracking stability, achieving an IDF1 of 97.89% and a MOTA of 95.79%. Compared to Top-K selection, our algorithm also increases visual diversity by 146%, temporal coverage by 89%, and information retrievability by 3.5%.