Generative AI changes mentoring in open source software programs
Mentorship, Interrupted: How GenAI Disrupts the GSoC Pipeline and Why Mentors Matter More
Software Engineering
Summary
Generative AI tools make it easier for newcomers to create open source software contributions, but these contributions reveal less about what the newcomers actually understand. The authors studied how this affects mentorship in the Google Summer of Code program, where mentors guide and evaluate new contributors. They found that AI can help early steps but makes judging newcomer skills harder, increasing mentors' workload. Different organizations reacted differently, and the authors suggest a framework to help decide how to deal with AI in mentoring.
What this means in practice
- •For open source project maintainers: Adjust mentoring practices to verify newcomer skills effectively while managing mentor workload when generative AI is involved.
- •For software development team leads: Integrate guidelines for acceptable AI usage in onboarding and contribution review to maintain team skill assessment quality.
Tested on one dataset.
Authors
Ruizhe Xing, Zixuan Feng, Mary Radomile, Stephanie Taylor, Sophia Vargas, Anita Sarma
Abstract
Generative AI (GenAI) makes open source software (OSS) contributions cheaper to produce, and it makes them say less about the people who produced them. In OSS mentoring, that matters: mentors judge what newcomers understand from their work, and newcomers learn from mentors' feedback. We studied how GenAI is changing mentoring in Google Summer of Code (GSoC), a program in which today's mentees can become tomorrow's mentors. We screened every thread on GSoC's restricted mentors' mailing list from November 2022 through March 2026 and interviewed 15 mentors and organization administrators. GenAI helped with onboarding, coordination, and first-pass review, but proposals, entry contributions, and mentees' answers became weaker evidence of what newcomers could do. Mentors made up the difference by checking, and that checking drew on the same time and goodwill the program needs to renew itself; one organization had withdrawn from GSoC. Responses ranged from banning GenAI to accepting it, and the boundary an organization drew determined who did the checking. We propose AIM, a decision framework that links acceptable GenAI use, integration into the mentoring pipeline, and mentoring capacity, to help organizations choose responses they can sustain.