Can temporal article-level credibility signals improve domain-level credibility prediction?
2026-07-06 • Computation and Language
Computation and Language
AI summaryⓘ
The authors address the problem of checking if new websites are trustworthy, which is hard because new sites don't have a reputation yet. They note that experts usually judge trustworthiness by looking at the content for misinformation or bias, but with lots of new content created quickly by AI, manual checking is not enough. To solve this, the authors created a system called Domain Credibility Evaluation Framework (DCEF) that uses expert ratings to automatically assess website trustworthiness by analyzing their articles over time, even without knowing the site's background beforehand.
web domain credibilitymisinformationfact-checkinglarge language modelsautomated evaluationdomain reputationbias detectioncontent analysistemporal frameworkexpert ratings
Authors
Islam Eldifrawi, Shengrui Wang, Amine Trabelsi
Abstract
Web domain credibility evaluation is vital for combating misinformation. It is conducted by examining factors such as domain type, transparency, and overall reputation. However, assessing the credibility of newly emerging web domains remains challenging since they have no reputation yet. Expert fact-checkers evaluate the credibility of domains by analyzing the content of their articles, including the presence of misinformation, bias, or propaganda. Yet, the ease of large-scale content generation enabled by LLMs has accelerated the creation of new content, rendering manual assessment insufficient and underscoring the need for automated approaches to domain credibility evaluation. In this paper, we introduce our Domain Credibility Evaluation Framework (DCEF), a temporal framework for domain credibility evaluation grounded in expert ratings. DCEF enables us to investigate whether the credibility of web domains can be assessed from their published articles following the workflow of expert fact-checkers, without any prior knowledge of the source domains themselves.