Self-claims in domain names can mislead AI safety checks
ClaimMirage: When Self-Claims in Domain Names Change LLM Threat Judgments
Cryptography and Security
Summary
Some website names include words like 'official' or 'not phishing' to make AI models think the site is safe, even if it's not. This can trick large language models into giving fewer warnings or more alerts, depending on how they're tested. The researchers showed that these self-claims can confuse AI security checks and suggest that independent evidence should back up any safety judgments about websites. Different AI models and settings respond very differently to these fake claims.
What this means in practice
- •For security software developers: Improve AI-based phishing detectors by testing and mitigating vulnerability to misleading self-claims in domain names.
- •For web hosting providers: Detect domain name registrations that misuse self-claim terms to appear safer than they are and flag suspicious registrations.
Authors
Daiki Chiba, Hiroki Nakano, Takashi Koide
Abstract
Short claims such as not-phishing or official can change how a large language model (LLM) judges a domain name, without explicit prompt-injection commands. We study this manipulation as ClaimMirage: a name under inspection claims its own safety or approval. We analyze 622,080 judgments across 64 brands and five LLMs, comparing ten claims with length- and hyphen-matched controls in constructed brand-like names. Self-claims can substantially reduce or increase alerts, depending on the LLM and input setting. In one setting, risk-denial terms inside the registrable name reduce alerts by 45.3 percentage points even with a basic safeguard: the prompt supplies the potentially impersonated brand and its official domain for comparison. Without these references, endorsement terms at that position increase alerts by 65.6 points in the same LLM. References and component annotation remove some alert reductions but leave others or make them larger. These findings motivate testing resistance to self-claims and seeking independent evidence before treating a domain name under inspection as safe or authorized.