Large language models improve detection of multiple authors writing styles
Multi-author style detection with stylistic embeddings
Computation and Language
Summary
Figuring out when a text changes from one writer to another can be tricky. The authors found that using big language models trained in one area doesn’t work well if applied directly to this problem. Instead, they trained models on a related task—confirming who wrote a text—and then used that learning to better spot style changes when multiple authors write together. By combining two different language models, they got even better at detecting these changes.
What this means in practice
- •For content moderation teams: Identify when multiple users contribute to a single piece of text in online forums or social media to detect coordinated behavior or plagiarism.
- •For digital forensics analysts: Use enhanced authorship style change detection to help determine whether documents have multiple authors in legal or investigative contexts.
Authors
Man-Ling Sung, Chak Fai Li, Damianos Karakos, Fred Bernardin, Brian Ulicny
Abstract
We investigate the use of Large Language Models (LLMs) in multi-author style change detection. Using off-the-shelf LLMs or supervised fine-tuning solely within the target domain fails to yield effective features; for this reason, we propose cross-domain transfer learning with LLM-based embeddings fine-tuned for authorship verification. We fine-tune Mistral-7B embedding model and compare it with DeBERTa, used in the state-of-the-art (SOTA) systems in PAN-25 multi-author writing style analysis challenge. Results show that DeBERTa and Mistral excel in different user-generated text scenarios. We exploit this diversity by combining both embeddings, leading to more effective style change detection.