Contrastive learning for video reuse detection

Computer Vision and Pattern Recognition

Summary

The gist is being written…

Authors

Tristan J. B. Cann, Arjun Biswas

Abstract

The incorporation of footage from other sources has long been a concern from a copyright perspective, as such usage can harm the interests of rights holders. Recent trends in the ``remixing'' of social media content and the development of bespoke stock libraries for specialist causes mean that methods to identify content reuse will have a range of applications. In this paper, we develop a framework based on a Siamese network fine tuned using contrastive learning to identify potential sources for key frames in a given video. This framework makes two contributions to analysis of large video datasets. Firstly, we demonstrate its utility at reducing complex video data to a set of key visuals typically less than 10\% of the size of the full set. Secondly, we show that these reduced sets allow for efficient and effective identification of component footage. We test this method over a set of videos compiled from a stock footage library and demonstrate its effectiveness for constituent video identification with average accuracy greater than 0.95.