The ICSE 2026 Shadow PC: Training the Next Generation of Reviewers Through Deliberate Practice
2026-07-22 • Software Engineering
Software Engineering
AI summaryⓘ
The authors explain a new program called the ICSE 2026 Shadow PC designed to train research paper reviewers more effectively. This program uses multiple stages, including feedback and calibration, and is kept separate from the main review committee to improve learning. With over 100 participants and positive feedback from both reviewers and authors, the authors show that structured training for reviewers is possible on a large scale. They also suggest using experienced shadow reviewers to help develop future leaders in the process.
peer reviewsoftware engineeringICSEreviewer trainingcalibrationpeer feedbackshadow programleadership developmentprogram evaluation
Authors
Christian Kästner, Michael Hilton, Jonathan Bell, Francisco Gomes de Oliveira Neto
Abstract
Peer review is essential to software engineering research, yet reviewer training remains largely implicit. We describe the ICSE 2026 Shadow PC, a redesigned program emphasizing deliberate practice at scale. Key innovations include multi-phase structure with calibration and peer feedback, strict separation from the main PC, and a pathway toward leadership development. With 102 participants completing the program and reviewing 117 papers, 97% recommending the experience, and positive reception from authors (67% finding reviews helpful), the program demonstrates that rigorous reviewer training is achievable at scale. We share lessons learned and propose shadow PC area chairs as a mechanism for sustainable scaling and leadership development.