Ontology-Guided Multi-Agent Extraction of Evaluation Objects from Academic Review Texts: Evidence from Chinese Library and Information Science
Computation and Language
Summary
The authors developed a new computer method to better find and classify evaluation-related terms in academic reviews and commentaries, which are usually hard to detect because they can be vague or overlap in meaning. Their approach uses a special guide called an ontology combined with multiple steps to discover and categorize these terms more accurately. Tests showed their method works much better than previous techniques, helping researchers use evaluative texts in a structured way for studying science and technology. The authors also found that their multi-step process and ontology rules improved both detection and classification quality.
Authors
Haolin Chen, Hongyi Dong, Yu Zhu, Yijia Hong, Leiqing Niu, Jiyuan Ye
Abstract
Academic reviews, scholarly commentaries, and book reviews serve as sources of evaluative statements about theories, methods, literature, institutions, and policies, providing valuable evidence for scholarly evaluation. Existing scientific entity extraction methods mainly target research articles and are less effective for evaluation objects, which are often abstract, context-dependent, and characterized by ambiguous type boundaries. This study proposes an ontology-guided multi-agent framework for evaluation object extraction. The framework combines candidate discovery, ontology-constrained classification, and domain review. Experimental results show that it achieves a Precision of 90.33%, Recall of 84.55%, Entity-level F1 of 87.34%, Strict Typed F1 of 79.78%, and Type Accuracy of 91.35%, substantially outperforming rule-based and zero-shot baselines. Ablation results indicate that the multi-agent workflow improves recall and stability, while ontology-based boundary constraints enhance fine-grained classification and reduce category confusion. The framework supports the structured utilization of evaluative scholarly texts and provides methodological support for evidence-based research evaluation and STI mining.