Chinese military news benchmark helps improve event and entity extraction
CMNIE: An Information Extraction Benchmark for Chinese Military News
Computation and Language
Summary
Extracting detailed, structured information from Chinese military news articles is hard because events, people, places, and their connections all need to be understood at once. The authors created a new dataset called CMNIE with 13,000 carefully labeled news instances that identify different event types, important roles in those events, named entities, and relations between them. They tested current AI models and found the task remains difficult, especially for exact matches of event details and relationships. This dataset creates a shared challenge to help improve how machines understand complex military news in Chinese.
What this means in practice
- •For intelligence analysts: Improve automatic extraction of military events and relationships from Chinese news to support faster situational understanding.
- •For knowledge base engineers: Build more accurate Chinese military knowledge graphs by jointly extracting events, roles, entities, and relations from news sources.
Authors
Yan Yu, Mengna Zhu, Zhenyu Song, Hao Yang, Haiwen Chen, Mao Wang
Abstract
Structured extraction from Chinese military news supports intelligence analysis, decision-making, and knowledge base construction. However, existing resources provide limited support for joint informa?tion extraction in this domain, especially when events, event arguments, entities, and relations must be modeled together. We present CMNIE, an information extraction benchmark for Chinese military news. Extend?ing military-domain resources beyond document-level event annotations, CMNIE jointly annotates event triggers, event arguments, named enti?ties, and entity relations under a unified domain schema. The dataset contains 13,000 instances collected from public Chinese military news, with manual annotations for 7 event types, 10 argument roles, 7 entity types, and 8 relation types. We evaluate supervised IE models, zero-shot large language models, and fine-tuned LLM-based extraction methods on a shared test set. Experimental results show that CMNIE remains chal?lenging, especially for relation extraction and exact matching of event?argument spans; zero-shot LLMs often identify relevant semantic units but fail to match gold span boundaries exactly. CMNIE provides a stan?dardized benchmark for studying schema adherence, exact span match?ing, and joint structured extraction in specialized Chinese news.