Riskchainbench measures hidden message decoding and web checks for platform threats
RiskChainBench: A Benchmark for Obfuscated Platform Message Restoration and Evidence-Grounded Web Investigation
Computation and Language
Summary
Online scams hide harmful instructions using tricky text tricks like emojis and symbols to mislead users toward dangerous websites. The authors created RiskChainBench, a set of tests that first sees if a model can correctly understand these hidden messages, then checks if it can investigate the linked websites properly to find risks. Their work helps show the challenges in fully recovering hidden directions and verifying associated risks in a reliable way. It also provides tools and a safe environment for testing and improving such systems.
What this means in practice
- •For cybersecurity teams: Detect and analyze hidden instructions in abusive platform campaigns to produce evidence-backed risk reports for threat response.
- •For web content moderators: Evaluate links and web environments from obfuscated messages to make more accurate content safety decisions.
Authors
ZhuoXin Liu, Zhiming Ma, Ying Zhang, Mengzheng Yang, Yifan Wang, Zhengqi Huang, Yanhan Zhou, Zekun Lin, Jun Zhang, Shun Zhang, Yue Chen, Qiao Zhao, Peng Chen
Abstract
Platform abuse campaigns conceal redirection instructions with emojis, homophones, character decomposition, and redundant symbols, then route users through disguised links to services associated with pornography, fraud, gambling, or illicit transactions. Existing benchmarks evaluate obfuscated text and risky webpages separately, obscuring how target recovery affects downstream evidence acquisition. We introduce RiskChainBench, pairing 3,600 synthetic token-text restoration inputs from 600 source sessions with 600 corresponding human-labeled local web environments. A model first restores the message, operational intent, and destination; the same underlying model then acts as a VLM-driven web agent that investigates the correctly associated website and produces a frozen, evidence-cited risk report without message-side semantics or domain-reputation cues. We score restoration and correct-routing web investigation separately and compose them offline by applying the frozen primary-entry prediction as a gate to the same Task 2 result. Human labels determine task correctness, while a fixed multimodal evidence judge assesses faithfulness, sufficiency, completeness, and consistency. Across ten models, Entry Top-1 ranges from 35.2% to 95.2% and web decision accuracy from 26.3% to 62.8%; the leading systems differ across entry recovery, full reconstruction, website decisions, and fine-grained typing. Execution failures account for 31.9% of web runs, whereas post-decision type errors account for only 0.9%, identifying stable exploration and risk judgment as the principal bottlenecks. We release the benchmark, protocol, and resettable local sandbox.