Designing an Auditable LLM-Supported Workflow for Qualitative Thematic Analysis

2026-08-31Artificial Intelligence

Artificial IntelligenceSoftware Engineering
AI summary

The authors show how large language models (LLMs) can help with qualitative research by turning a well-known method called Thematic Analysis (TA) into a computer process that is transparent and respects privacy. They created a clear workflow that uses LLMs to identify themes in interview texts while keeping track of where each idea comes from. Their tests on Danish interviews found the computer’s work was similar to human analysis but with fewer, broader themes. This method can be adjusted for different LLMs and research topics by changing how the models are prompted.

Large Language ModelsThematic AnalysisQualitative ResearchInductive AnalysisLatent ThemesInterpretative ContextComputational WorkflowPrivacy-PreservingPrompting StrategyAnalytical Justification
Authors
Nadia Jul Jeldtoft, Tariq Yousef
Abstract
Large Language Models (LLMs) offer new possibilities for scaling qualitative analysis, but existing applications often provide limited methodological transparency regarding how qualitative methods are translated into computational procedures. This paper presents an auditable and privacy-preserving computational operationalization of inductive and latent Thematic Analysis (TA). This paper first derives five design principles from the methodological requirements of TA and the conditions introduced by LLM-based inference: preserving interpretative context, maintaining traceable relationships between empirical material and analytical outputs, representing analytical constructs and reasoning explicitly, constraining LLM inference to interpretative tasks, and enabling privacy-preserving local deployment. Second, it presents a proof-of-concept for a two-phase workflow that operationalizes these principles by combining interpretative LLM inference with deterministic procedural control to generate codes, analytical justifications, themes, and theme descriptions while preserving explicit links to the source material. Third, it proposes an evaluation framework combining structural comparison with human-led TA and independent expert assessment of analytical quality. The evaluation is conducted on semi-structured Danish interview transcripts. and the results shows that the workflow produces code-level outputs with coverage broadly comparable to human annotations and highly rated analytical justifications, while generating a more compressed thematic structure characterized by fewer and broader themes. The findings demonstrate the feasibility of auditable LLM-supported TA through a modular workflow designed to scale to larger datasets, accommodate different LLMs, and support transfer across research domains, with domain adaptation primarily requiring adjustments to the prompting strategy.