Large language models show promise for evaluating trust in cybersecurity
LLM4Trust: Exploring the Capabilities of Large Language Models for Trust Evaluation
Machine Learning
Summary
Trust evaluation helps computers decide who and what to trust, which is important for cybersecurity safety. Current methods require lots of data and are hard to explain, so the authors tested if large language models (LLMs), which understand language well, could do this better. They created tests to see if LLMs understood key trust concepts and found that some worked well, especially when given little training. However, these models can be tricked and are costly to run, so the authors suggest ways to make them safer and more efficient.
What this means in practice
- •For cybersecurity analysts: Improve automatic assessment of trust relationships in security networks with minimal labeled data.
- •For social network platform engineers: Detect and evaluate trustworthiness in user connections using large language models to reduce manual intervention.
Authors
Jie Wang, Yanbo Sun, Zheng Yan, Jiahe Lan, Elisa Bertino
Abstract
Trust evaluation plays a critical role in cybersecurity by supporting risk mitigation and decision-making. A variety of trust evaluation methods have been proposed, with learning-based approaches offering high accuracy and automation. However, they often require substantial ground truth, suffer from low training efficiency, lack support for basic trust properties, and provide limited explainability. Large Language Models (LLMs) offer a compelling alternative due to their strong zero-/few-shot reasoning abilities and broad knowledge. To this end, we propose LLM4Trust, the first benchmark framework that systematically explores the capabilities of LLMs for trust evaluation. We first construct diverse trust graphs to model five basic trust properties and design corresponding property understanding tasks. We then assess the ability of eight representative LLMs to understand these properties under nine prompt methods. Based on this exploration, we identify the most effective LLM-prompt combinations and apply them to five real-world datasets for validating LLMs' trust evaluation capability. During this process, we propose two strategies to extract key information from large-scale trust graphs, addressing the context window limitations of LLMs. Extensive experiments show that LLMs can effectively understand basic trust properties and have great potential for real-world trust evaluation, particularly under limited supervision. However, they remain vulnerable to attacks targeting trust graphs and demonstration examples used in few-shot prompting, and incur high inference costs. Accordingly, we propose a defense mechanism and batch inference to improve the robustness and efficiency of LLM-based trust evaluation. The source code of LLM4Trust is available at https://github.com/Jieerbobo/LLM4Trust