What Does It Take to Research with AI? A Rapid Review of Competencies to Train LLM-Literate Researchers
Software Engineering
Summary
The authors reviewed many articles to understand what skills researchers need to use large language AI tools responsibly. They found eight key skills, with the most important being deep knowledge of the research field and careful checking of AI outputs. Other important skills include thinking about when to use AI, ethics, crafting good prompts, and making research reproducible. The authors emphasize that just knowing how to use AI technically is not enough; researchers must also take responsibility for the knowledge AI helps produce. These insights can help improve training for graduate students and researchers.
Authors
Danilo Monteiro Ribeiro, Ronnie de Souza Santos, Rodrigo Siqueira, Breno Andrade, Rafael Batista Duarte, Gilberto Hida, Julia Alencar, Gustavo Pinto
Abstract
The growing adoption of Large Language Models in scientific research has created a need to understand what competencies researchers and graduate students require to use these tools critically and responsibly. This rapid review analyzed 194 articles retrieved from Elicit and Google Scholar (2022 to 2025), from which 40 were selected for competency extraction and thematic analysis following independent dual screening (Gwet AC1: 0.76 to 0.83). Eight competencies were identified. The most prevalent was domain expertise and oversight of AI outputs (n = 123), encompassing subject matter mastery, systematic skepticism, source verification, and researcher accountability. Other key competencies include metacognition and decision making about AI use (n = 55), ethics and academic integrity (n = 53), prompt engineering for research (n = 38), and reproducibility of AI use (n = 29). AI literacy and technical knowledge (n = 16) was explicitly identified as a risk factor when absent, with domain expertise treated as a prerequisite for meaningful critical evaluation. The findings suggest that preparing researchers to use LLMs goes beyond technical instruction, requiring an integrated set of epistemic, ethical, and methodological competencies centered on human accountability for the knowledge produced. These results have direct implications for the design of graduate programs and AI literacy initiatives.