ECLAIR: A Causally-Grounded AI Framework for Scientific Discovery in Empirical Software Engineering
2026-08-03 • Software Engineering
Software Engineering
AI summaryⓘ
The authors present ECLAIR, a new framework that uses AI language models to help with all parts of scientific research in Software Engineering. They treat these AI models as careful scientific helpers that follow rules of cause and effect, with humans making sure decisions are sound. In a test case, they showed how different prompt styles affect the AI's ability to write correct code, finding some prompt types slightly reduce accuracy. This work aims to make AI-assisted research more reliable and understandable by grounding it in causal reasoning.
Software EngineeringScientific MethodLarge Language ModelsCausal InferenceHuman-in-the-loopPrompt DesignCode GenerationEmpirical ResearchArtificial Intelligence
Authors
Alejandro Velasco, Daniel Rodriguez-Cardenas, Dipin Khati, David N. Palacio, Denys Poshyvanyk
Abstract
The scientific method has long guided empirical research in Software Engineering (SE), but the complexity of modern software systems often hinders its systematic application. This paper introduces _ECLAIR_, a causally grounded AI framework that integrates Large Language Models (_LLMs_) into every stage of the scientific process, from hypothesis generation to analysis and interpretation. _ECLAIR_ treats _LLMs_ as active **scientific agents** operating under the principles of causal inference, within a human-in-the-loop design that safeguards against the risks of unsound automated reasoning. We demonstrate the framework through a case study examining how prompt design influences code generation accuracy in two _LLMs_. Results show that, for both models, instruction-style, longer few-shot, and signature-augmented prompts yield small negative causal effects on accuracy, illustrating how causal reasoning provides a principled foundation for explaining _why_ software phenomena occur. This study presents the first causally grounded structured methodology for embedding _LLMs_ within the scientific method in SE, designed around the epistemological demands of empirical SE research, establishing a basis for rigorous AI-assisted research.