Papers for

computational workflows developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Benchmarks reveal scaffold effects and execution checks in autonomous materials simulation

DFT-Agent Bench: Scaffold Effects and Execution Provenance in Autonomous DFT

Abstract: Large language models are increasingly deployed as autonomous agents for first-principles materials simulation. We report empirical findings from DFT-Agent Bench, a benchmark of 200+ Quantum ESPRESSO tasks spanning relaxation, electronic structure, equation-of-state, and elastic-tensor pipelines over multiple material tiers. Each task carries a QE-derived ground truth with public-reference cross-checks where available, and a provenance audit checks whether the required QE execution contract was met. Under one execution harness, we compare a near-bare ReAct baseline with a bundled v5.1 scaffold on two models. The complete scaffold improves bands+DOS by +0.08/+0.11 and d-electron vc-relax by +0.17 on both models, while high-scoring relaxation and equation-of-state cells leave little room for improvement. The elastic result is model-dependent (0.18 to 0.40 for deepseek-v4-pro and 0.62 to 0.59 for claude-opus-4-8). A fixed 100-task analysis with retrieval disabled supports the aggregate pattern: the fully frozen DeepSeek comparison gains +0.17, while the Opus comparison gains +0.10 against a historical pool baseline (p < 0.001). The provenance gate catches incomplete or unmarked executions that can receive high rubric scores; it is an execution-contract check, not a test for intentional fraud. These results characterize the evaluated DFT workflows. The anonymous review artifact contains the frozen diagnostic slice, prompts, scorer, and solver revisions: https://anonymous.4open.science/r/231e31re91287h/README.md.

Sat 7 NovArtificial Intelligence
The gist
Simulating materials at the atomic level helps scientists understand their properties, but these simulations are complex and require careful computer instructions. The authors tested different ways that AI language models guide these simulations, comparing a simple baseline with a more detailed approach called a scaffold. They found the scaffold improved accuracy for many tasks and also developed a way to check that the computer actually ran the full simulation. Their work helps show how AI can better assist in these materials simulations.
Open → 2611.10629v1