StreamProfileBench: A Benchmark for Fine-Grained User Profile Inference in Real-World Streaming Scenarios
2026-05-25 • Computation and Language
Computation and Language
AI summaryⓘ
The authors created StreamProfileBench, a new test to see how well language models can keep track of users' changing interests over time using streams of new posts. They gathered a large dataset from real users across multiple platforms to mimic how user interests evolve continuously. Their tests showed that current models struggle to update profiles correctly, often clinging too much to old interests and missing when interests fade. This highlights the need for better methods that handle ongoing updates rather than just static snapshots.
Large Language ModelsUser ProfilingStreaming DataUser-Generated ContentTemporal CorrelationContinuous State MaintenanceBenchmark DatasetInterest DecayModel EvaluationFine-grained User Profiling
Authors
Sizhe Wang, Feiyu Duan, Juelin Wang, Liwen Zhang, Feiyu Duan
Abstract
Large Language Models (LLMs) have reshaped user profiling, yet current evaluations mainly focus on static data snapshots. This paradigm overlooks the reality of personalized systems, where User-Generated Content (UGC) arrives continuously and fine-grained profile evolve rapidly. To bridge this gap, we introduce StreamProfileBench, a large-scale benchmark for fine-grained streaming user profiling. We formalize streaming user profiling as a continuous state maintenance task and curate a highly authentic dataset comprising over 120,000 UGC posts from 7,000+ real users across five diverse platforms. By leveraging the temporal correlation of user interests, we further propose a novel, annotation-free evaluation framework. Extensive experiments across 14 leading LLMs reveal that continuous profile updating remains an open challenge. Models exhibit a systemic conservative bias, over-retaining past interests while failing to recognize interest decay. Ablation experiments further validate the practical utility and necessity of the streaming paradigm.