Verusweave benchmark tests fixing proofs in concurrent rust code

VerusWeave: A Benchmark for Proof Repair in Concurrent Verus Programs

Software EngineeringArtificial IntelligenceLogic in Computer ScienceProgramming Languages

Summary

Concurrent software, like the kind inside operating systems, is tricky because threads can cause subtle bugs. The authors created VerusWeave, a set of programming puzzles that test if automated tools can fix missing or broken logical proofs in verified Rust code handling concurrency. These puzzles come from real projects and focus on connecting low-level operations with the bigger correctness rules. Their tests show fixing entire modules automatically is still hard, making VerusWeave a useful tool for improving such automation.

What this means in practice

  • For software engineers: Evaluate and improve automated tools that repair correctness proofs in concurrent Rust programs, preventing subtle threading bugs.
  • For distributed systems developers: Test automated proof repair methods that maintain correctness across thread interactions in low-level concurrent modules.

Authors

Pedro Ribeiro, Martim Afonso, André Vasconcelos, João F. Ferreira

Abstract

Concurrent software underpins critical infrastructure, including operating systems and databases, yet thread interactions can have bugs that bypass conventional testing. Verus extends Rust with formal verification capabilities, enabling developers to specify, implement and prove the correctness of concurrent code in the same language. Although coding agents have been explored for automated proof construction, existing Verus benchmarks offer limited coverage of concurrency-specific examples that connect atomic operations to abstract protocols. To address this gap, we introduce VerusWeave, the first benchmark dedicated to concurrent proof repair, derived from four open-source projects. It comprises 75 focused tasks and 17 full-module variants that exercise features used to verify low-level concurrent code. Tasks remove selected proof annotations and require agents to fill gaps, while preserving executable code and specifications. The evaluation harness validates repairs using Verus and a change checker, recording solutions, execution traces, and resource usage. An initial evaluation of four general-purpose coding-agent harnesses achieved 75-85% success on focused tasks and 29-47% on full-module variants, highlighting the challenge of autonomous module-wide repairs. VerusWeave provides a reusable foundation for evaluating automated proof construction for concurrent software.