Papers for

media monitoring organizations

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Comparison shows key differences in global news data sources

Comparison of Common Crawl News & GDELT

Abstract: The corpus of worldwide news is important for natural language processing, knowledge graphs, large language models, and other technical efforts. Additionally, this corpus is important for understanding the people, places, organizations, and events that interact in real-time every day. This paper compares two news datasets used for these tasks today, namely the Global Database of Events, Language, and Tone (GDELT) and Common Crawl News. Our research highlights the strengths and limitations of each dataset, analyzing their content and coverage. Notably, while GDELT relies on broadcasts, prints, and web news from across the globe, Common Crawl focuses on news sites from around the world gathered through web crawling. Our analysis revealed considerable differences in where the two datasets gather their news sources.

Wed 30 SeptInformation Retrieval
The gist
Having a big collection of news stories from around the world helps computers understand what's happening everywhere. This paper looks at two popular collections of news called GDELT and Common Crawl News. The authors found that these collections get their news from different kinds of places—GDELT collects from broadcasts, printed media, and web news, while Common Crawl gathers news mainly by scanning many websites. This difference means they cover the news world in different ways.
Open → 2610.00587v1