Babbar26 dataset labels web pages by content type and language
Babbar26 main x genre
Information Retrieval
Summary
Web pages have different types, like news articles or blogs, and they contain main content mixed with extra stuff like ads. The authors created a big, multilingual set of over 1100 web pages from five languages, labeling them by kind and separating important content from noise. They made sure labelers agreed well on their work. Their set has more complex pages than older similar collections, making it useful for testing web content analysis.
What this means in practice
- •For web developers: Use Babbar26 to train tools that detect main content and filter out boilerplate on multilingual websites.
- •For digital marketers: Rely on Babbar26 to better classify web pages by genre for targeted advertising across different languages.
Authors
Bournonville Aur{é}lien, Clotilde Levy, Thomas Largillier, Guillaume Pitel, Fran{\c c}ois Rioult, Fabrice Maurel
Abstract
We introduce Babbar26, a multilingual dataset of 1111 web pages from 1006 domains in five languages (English, French, German, Spanish, Italian), jointly annotated for genre, following a compact five-class taxonomy, and for main content and boilerplate, as main/noise regions anchored to the DOM. We describe the annotation guidelines and report double-annotation agreement on both tasks (97.1% on genre; content-level Cohen's k = 0.766 for main content). Compared with the classic KI-04 genre benchmark, a median page has 2.3x more DOM elements and 1.9x more readable text.