Large language models assist automated privacy impact assessments
PIA-Bench: Towards Automated Privacy Impact Assessment with Large Language Models
Cryptography and Security
Summary
Privacy impact assessments help organizations find and fix privacy problems before launching new systems, but they usually need experts. The authors created a benchmark called PIA-Bench to test how well large language models (LLMs) can do this job by looking at real-world government assessments. They found that current LLMs can produce useful privacy risk identifications and suggest ways to reduce those risks, although there is room for improvement. Their work suggests ways to improve how AI tools are used for privacy checks and calls for better standards and accountability.
What this means in practice
- •For privacy compliance teams: Automatically evaluate and identify privacy risks in complex systems using LLMs to support compliance workflows in organizations.
- •For software security engineers: Use LLM-generated privacy risk assessments and mitigations to anticipate privacy challenges during system design and development.
Authors
Jiamin Zheng, Hao-Ping Lee, Luo Mai, Jingjie Li
Abstract
Privacy impact assessment (PIA) is a critical instrument for institutions to proactively identify privacy risks and develop mitigation strategies before system deployment. While mandated across regulatory and institutional contexts, executing PIA requires extensive privacy and technical expertise, posing a particular challenge for teams without access to such resources. Prior work shows the potential of leveraging large language models (LLMs) to assist practitioners' privacy decisions, but little is known about how accurately and reliably LLMs can automate PIA. To this end, we develop PIA-Bench, the first open benchmark for evaluating LLMs on real-world PIAs. We first audited 499 expert-authored PIAs published by US federal agencies and curated 73 structured PIAs, comprising a total of 451 privacy risk and 831 mitigation items, to evaluate LLMs' ability to assess privacy risks and propose mitigations of complex systems. Our results show that off-the-shelf LLMs produce meaningful assessments and identify avenues for future improvement. Finally, we call for improving domain-specific workflows for LLM agents, developing accountable LLM infrastructure, and designing new quality standards for PIAs.