Papers for

nlu developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Llm methods improve entity conflict handling in niche nlu systems

Can LLMs Resolve Entity Conflicts in Domain Specific NLU Pipelines?

Abstract: Resolving entity conflicts is a critical step in Pipeline-based NLU architectures, especially when multiple extractors provide overlapping or conflicting outputs. Traditional rule-based entity conflict resolution methods often lack flexibility and are unscalable. In this work, we aim to study if the broad knowledge of LLMs can be leveraged to resolve entity conflicts in domain specific NLU pipelines. We present an experimental comparison between a traditional rule-based conflict resolution method, and two LLM-based approaches: a multi-shot prompting method using the Llama3.1-8B-Instruct model and a LoRA fine-tuned Llama3.1-8B-Instruct model. The evaluation was conducted under two setups: (1) a broad assessment of LLM's capability to match existing NLU pipeline output, and (2) a more controlled scenario assessing the LLM accuracy on human-labeled data. Results show that while multi-shot prompting offers ease of adaptation and rapid behavioral changes, the fine-tuned model delivers superior performance on complex, ambiguous cases. Ultimately, the best approach appears to be a combination of both strategies, leveraging multi-shot prompting's flexibility alongside the broad accuracy gains of LoRA fine-tuning for a more comprehensive performance.

Thu 19 NovInformation RetrievalMachine Learning
The gist
When computers try to understand language, they often find overlapping or conflicting information about things called entities, like names or dates. The authors studied whether large language models (LLMs) can better solve these conflicts than traditional rule-based methods, especially in specialized language understanding systems. They tested two LLM approaches: one using examples to guide the model and another where the model was fine-tuned for the task. Their results show that using both methods together leads to the best performance, combining flexibility with accuracy.
Open → 2611.25733v1