The CRAFT principles for the responsible use of large language models in policymaking
Computers and Society
Summary
The authors explain that large language models (LLMs) can help policymakers by making it easier to gather and understand important information and by helping write policy documents. However, LLMs also come with risks like giving wrong but believable answers, showing biased views, leaking sensitive data, and making people rely too much on them. To use LLMs safely in policymaking, the authors suggest following the CRAFT principles: control, rigour, accountability, fairness, and transparency. These principles aim to balance the benefits and risks of LLM use in government decisions.
Authors
Willem Fourie, Gray Manicom, Tanya de Villiers-Botha
Abstract
Policymakers around the world face the question of how to use artificial intelligence in general, and large language models in particular, to improve the policymaking process. Used well, large language models can strengthen the collection, interpretation and synthesis of policy-relevant information and the drafting of policy-relevant output. Yet the use of large language models in policymaking is associated with risks. Output that is plausible but not necessarily correct, bias resulting from unrepresentative training data, the exposure of sensitive information and, over time, deskilling and dependency can erode trust if large language models are not used thoughtfully. The CRAFT principles - control, rigour, accountability, fairness and transparency - offer a way to make the most of large language models in policymaking while managing the risks.