Dualview improves multipoint text scoring with fewer parameters
DUALVIEW: Leveraging Broad and Output-Specific Evidence for LLM-based Multi-Output Text Regression
Computation and Language
Summary
Assigning multiple scores to text using large language models is hard and usually needs lots of training. The authors created DualView, a method that uses both general and specific information from a large language model without changing its core. DualView produces multiple related outputs efficiently and performs better than other training methods, even when data is limited. This approach works well with different models and tasks.
What this means in practice
- •For natural language processing engineers: Build systems that predict multiple related scores from text more accurately using smaller fine-tuned parameter sets.
- •For automated content moderation teams: Improve automated evaluations of text by generating multiple output scores efficiently and reliably, even with limited labeled data.
Authors
Yiheng Su, Junyi Jessy Li, Matthew Lease
Abstract
While work on LLM-based regression has largely focused on scalar prediction, multi-output text regression remains less explored. We present DualView, a parameter-efficient approach to leverage LLMs for multi-output text regression. \method{} first extracts broad and output-specific representations from a frozen LLM, then uses a dual-branch architecture to produce the full score vector. Across five heterogeneous regression tasks and eight LLMs (up to 122B), DualView outperforms seven supervised regression baselines while requiring 82--1842 times fewer fitted parameters than the three strongest baselines. These improvements extend across individual output dimensions and persist even with limited training data. Notably, DualView using a smaller open-weight LLM can outperform unsupervised prediction with Gemini 3.1 Pro. Controlled ablations show that broad and output-specific evidence provide complementary predictive benefits, while lightweight shared predictors achieve strong performance with fewer trainable parameters.