Dualview improves multipoint text scoring with fewer parameters

DUALVIEW: Leveraging Broad and Output-Specific Evidence for LLM-based Multi-Output Text Regression

Computation and Language

Summary

Assigning multiple scores to text using large language models is hard and usually needs lots of training. The authors created DualView, a method that uses both general and specific information from a large language model without changing its core. DualView produces multiple related outputs efficiently and performs better than other training methods, even when data is limited. This approach works well with different models and tasks.

What this means in practice

Authors

Yiheng Su, Junyi Jessy Li, Matthew Lease

Abstract

While work on LLM-based regression has largely focused on scalar prediction, multi-output text regression remains less explored. We present DualView, a parameter-efficient approach to leverage LLMs for multi-output text regression. \method{} first extracts broad and output-specific representations from a frozen LLM, then uses a dual-branch architecture to produce the full score vector. Across five heterogeneous regression tasks and eight LLMs (up to 122B), DualView outperforms seven supervised regression baselines while requiring 82--1842 times fewer fitted parameters than the three strongest baselines. These improvements extend across individual output dimensions and persist even with limited training data. Notably, DualView using a smaller open-weight LLM can outperform unsupervised prediction with Gemini 3.1 Pro. Controlled ablations show that broad and output-specific evidence provide complementary predictive benefits, while lightweight shared predictors achieve strong performance with fewer trainable parameters.