Interactive interviewer adjusts questions to participant expertise
Evidence-Traceable Dynamic Interviewer Architecture for Expertise-Adaptive Qualitative Interviews Using Local LLMs
Artificial Intelligence
Summary
Interviews usually follow fixed questions that don't change based on what the person knows, which can get boring or confusing. The authors designed a smart interviewer that listens to answers and adjusts the questions to fit the person's knowledge level. It runs on a locally stored language model and keeps track of the conversation to make the interview feel smooth and relevant. They tested it with 246 people and found it did a good job guessing expertise and asking fitting questions, while people found the experience engaging and useful.
What this means in practice
- •For recruitment teams: Create interviews that adapt question depth and style to candidates’ expertise in real time using a local language model.
- •For customer service managers: Deploy conversational agents that personalize follow-up questions based on users’ knowledge to improve engagement and satisfaction.
Authors
Aisvarya Adeseye, Jouni Isoaho, Adeyemi Adeseye, Seppo Virtanen, Mohammad Tahir
Abstract
Automated interviewers and conversational agents are increasingly used in research, recruitment, customer service, and education. However, many existing systems rely on fixed question sequences and provide limited context-based personalization without considering participants' knowledge, which can lead to repetitive or irrelevant follow-up questions. Therefore, there is a need for an adaptive interviewing system that can adjust question depth while maintaining conversational continuity and semantic progression. To address this, an Evidence-Traceable Dynamic Interviewer Architecture is presented using a locally hosted Large Language Model (LLM), with the interview continuously adapted throughout the entire conversation based on the participant's responses and evolving context. The interviewer profiles participants' expertise in real time to generate knowledge-appropriate questions, well-articulated responses, and smooth transition messages that support conversational continuity. A five-module prompt-driven architecture and persistent interview-state record support these functions. The interviewer was evaluated with 246 participants. Expertise Profiling module (M3) showed 78.9% exact agreement with independently reported participant expertise, with a weighted Cohen's K of 0.80. Generate Iterative Questions module (M4) showed a strong expertise-complexity association (p=.79, p<.001), and participants reported high relevance (mean 4.41), engagement (mean 4.32), and satisfaction (mean 4.38), providing evidence that the architecture's adaptive components operated consistently with their intended functions while participants reported a positive interview experience.