Papers for

economic research teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Model audits climate policy studies to find weak evidence support

Evidence-Grounded Auditing of Identification Assumptions in Climate-Policy Causal Evaluations

Abstract: Difference-in-differences (DID) studies are widely used to evaluate climate policy, but assessing the evidence supporting their identification assumptions remains challenging. We introduce ARGUS, a structured language-model pipeline that audits reported evidence against an eleven-dimension assumption-implication-evidence rubric and abstains when relevant evidence cannot be retrieved. We evaluate ARGUS using injected flaws, economics papers, and a small pilot with reconciled labels. On the 11-flaw benchmark, ARGUS detects 73% of planted flaws, compared with 18% for a keyword-based pipeline. Across 26 economics papers, ARGUS abstains on about 40% of paper-dimension assessments for lack of retrievable evidence. In a five-paper pilot with labels reconciled by two annotators, it assigns a higher risk level than the labels on 25 of the 33 assessments it completes. A rule fixed before the labels arrived removes most of this in-sample; weighted agreement stays low. ARGUS provides evidence-linked risk reports that localize potential weaknesses for expert review, without adjudicating causal claims. Code and data: https://github.com/yonghongzhang-io/ARGUS

Fri 25 SeptComputation and Language
The gist
Evaluating how climate policies work often uses a method called difference-in-differences, but it's hard to check if the studies' key assumptions are well supported by evidence. The authors created ARGUS, a tool that uses language models to carefully check if the required evidence is actually reported in a study, flagging many potential problems and sometimes saying it can't find enough evidence to decide. Compared to simpler approaches, ARGUS catches more flaws and highlights where experts should review the findings more closely. It helps highlight weak points without making final judgments on the policy effectiveness itself.
Open → 2609.30867v1