Personalized prompts improve document extraction for enterprise users
One Prompt Does Not Fit All: Self-Meta-Evolve for Personalized Information Extraction
Artificial Intelligence
Summary
When computers read documents for work, different users often want the information organized in their own unique ways. The authors point out that using one single prompt for everyone doesn’t work well because people have different needs. They develop a system called Self-Meta-Evolve that creates and continually updates a special prompt for each user based on their feedback. This approach helps get better answers faster and works well both in tests with simulated users and when real professionals try it.
What this means in practice
- •For enterprise software developers: Customize document information extraction for each employee to improve accuracy and satisfaction using adaptive prompts.
- •For customer support teams: Tailor automated ticket summarization and categorization to match diverse agent preferences and workflows with personalized prompt updates.
Authors
Hongliang Li, Lu Wang, Yong Xu, Hanyang Chen, Zhitao Hou, Xiaoting Qin, Song Ge, Qingwei Lin, Dongmei Zhang
Abstract
Large language models (LLMs) are increasingly deployed for enterprise information extraction (IE), where the same document must be reorganized differently for each user. Existing prompt optimization methods, however, rely on a single prompt optimized against a global objective, which is misaligned with the inherent user heterogeneity of real workplaces. We formulate enterprise IE as per-user prompt adaptation under interaction feedback and propose Self-Meta-Evolve, a hierarchical framework that maintains a dedicated prompt for each user and continuously refines it through a dual-loop process: an inner loop that edits structured prompts based on persona-conditioned feedback, and an outer loop that evolves the meta-prompt itself by distilling successful editing patterns. To enable scalable training and evaluation, we release a persona-driven IE benchmark of 292 simulated enterprise users, paired with a reproducible persona-generation pipeline grounded in O*NET occupational taxonomies. On this benchmark, Self-Meta-Evolve achieves a 74.58% success rate, outperforming the strongest prompt-optimization baseline by 13.56 absolute points, and reaches 52.54\% within only two iterations. A double-blind human study with twenty real professionals further confirms that prompts adapted by our framework win against static baselines in 71% of pairwise comparisons.