Specification helps trace fairness but not enforce it in AI coding agents
Specification-Driven Governance for Fairness-by-Design in Agentic Software Engineering: How Far Can We Go?
Software Engineering
Summary
Developers are using AI agents to write software, but making sure these agents keep fairness in mind is tricky. The authors tested whether clear fairness rules can guide these coding agents when building machine learning systems. They found that while agents almost fully capture and include fairness rules in their code, the resulting AI models don't actually become much fairer. This happens because agents mostly check fairness after the model is built instead of ensuring fairness during the training. The study shows tracking fairness is easier, but actually enforcing it needs better rules and workflows.
What this means in practice
- •For machine learning engineers: Use specification-driven approaches to document and trace fairness requirements throughout automated ML development workflows.
- •For software governance teams: Develop workflows that explicitly manage fairness decisions delegated from AI coding agents to human developers.
Authors
Alfonso Cannavale, Alessandra Parziale, Valeria Pontillo, Patrizio Pelliccione, Andrea De Lucia, Fabio Palomba
Abstract
Agentic AI is shifting developers' responsibilities from software implementation to software governance. While this paradigm improves productivity, it also raises challenges in ensuring that autonomous coding agents preserve developers' intentions. Among these, operationalizing fairness-by-design is largely unexplored, as existing approaches provide limited support for representing and propagating fairness requirements throughout agentic development workflows. In this paper, we investigate whether specification-driven development can operationalize fairness-by-design in the agentic development of machine learning systems. We study three coding agents on four machine learning applications, using four prompts that progressively operationalize fairness through nine Fairness Specification Elements, and analyze the traceability of fairness requirements across the generated artifacts, the fairness-accuracy trade-off of the resulting models, and the agents' design decisions and rationales. The results show that when fairness is operationalized, agents capture almost all fairness requirements in the specification (97.22%) and preserve most of them in the code (82.41%). However, the resulting models are not significantly fairer, as agents implement fairness as a post-hoc audit that neither shapes training nor prevents model delivery. Specifications thus make fairness traceable, but not enforceable, calling for fairness specifications that state what agents should do when requirements are not met, and for workflows that make the fairness decisions agents delegate to developers explicit.