Summary
Code generation AI models are used to write software, but it's hard to know if they were made ethically, including considerations like how they affect people and the environment. The authors created ES-Trace, a way to check how well companies share information about these ethical aspects, by looking beyond just the main summary documents called model cards. They found that many important social and labor details are missing, and that looking at all related documents improves the understanding of disclosures but also reveals inconsistencies. Their work suggests better, clearer ways to link different documents to accurately track ethical information about AI models.
What this means in practice
- •For software development teams: Assess ethical sourcing disclosures across all documentation to ensure responsible use of code generation models in projects.
- •For ai product managers: Improve transparency strategies by tracking and correcting inconsistencies in model version disclosures beyond model cards.
Abstract
Code generation models have been increasingly used in software development, but their development raises ethical-sourcing concerns involving intellectual property, privacy, fairness, labour practices, and environmental impact. Although prior work has defined ethical-sourcing criteria for code generation, it remains unclear how much evidence existing models disclose and where that evidence can be found. We introduce ES-Trace, a framework for ethical-sourcing disclosure audits that traces disclosed evidence beyond model cards using the Model Documentation Traceability Graph (MDTG), which represents relationships among models, versions, and documentation artifacts. We apply ES-Trace to 26 models from 10 publishers across 77 documents and 20 ES-CodeGen aspects. Model-card-only auditing yields a mean score of 1.77/5, while resolving the declared references increases it to 2.82/5, with most of the increase arising from documents that the publisher declares in structured metadata. The key findings of our study include: (1) social and labour-related aspects remain poorly documented, even when expanding the audit to the full documentation scope, (2) resolving documentation references substantially increases observed disclosure, raising the mean score from 1.77/5 to 2.82/5, (3) model-card-only audits can mischaracterize release-level disclosure changes, and (4) documentation mismatches can associate evidence with the wrong model or version, highlighting the need for explicit model--version binding and consistency across documentation artifacts. Our study calls for reference-aware ethical-sourcing disclosure audits, explicit model--version binding and consistency across documentation artifacts, and stronger documentation of currently underreported social and labour-related aspects.