Large language models enhanced with clear stepwise decision making
Integrating the Analytic Hierarchy Process with Large Language Models for Transparent Multi-Criteria Decision-Making
Artificial Intelligence
Summary
Decisions made by large language models can be hard to understand or trust, especially when these models explain their choices in ways that are not clear. The authors combined these models with the Analytic Hierarchy Process, which is a step-by-step method for making choices by comparing different factors carefully. They created a new dataset to teach and test the models on this method and found their approach better matched expert opinions in complex areas like legal cases and university rankings. This work helps make AI decisions more transparent and reliable.
What this means in practice
- •For legal support teams: Provide transparent and explainable multi-criteria analyses for complex legal decision tasks.
- •For university administration analysts: Generate interpretable rankings of educational institutions using a consistent stepwise decision framework.
Authors
Han Zhiguang, Farah Benamara, Pascale Zaraté
Abstract
LLMs are increasingly employed in a wide range of decision-making tasks. However, the opacity of their internal reasoning makes it difficult to validate or interpret their outputs, and the need for interpretability becomes especially critical in high-stakes settings. This study examines the decision-making capabilities of LLMs through the Analytic Hierarchy Process (AHP), a classical and widely used multicriteria decision-making framework. We construct a new annotated benchmark based on AHP and propose the first end-to-end approach that enables LLMs to perform the complete AHP workflow. Experiments in real-world decision problems in the legal and higher-education ranking domains show that our method significantly improves alignment with expert judgments.