Language model improves reasoning about others minds with evolving rules
CogRSI: Evolving and Auditing Cognitive Reasoning Strategies
Computation and Language
Summary
Figuring out what others know or think can be tricky, and computers struggle with this too. The authors created a system called CogRSI where a language model learns rules to track who knows what in stories. These rules can improve by checking their own accuracy on test questions. However, even if the rules help in practice, testing showed some of them still make errors in certain situations, meaning good practical performance doesn’t guarantee perfect understanding. This suggests both practical checks and detailed audits of reasoning rules are important.
What this means in practice
- •For natural language developers: Improve AI systems that track characters’ knowledge in stories using evolving rules learned from accuracy feedback.
- •For conversational ai builders: Enhance chatbot reasoning about what users know or believe by adopting adaptable cognitive rules tested against counterfactuals.
Authors
Yandan zheng, Haoran Luo, Xinliang Zhou, Anhtuan Luu
Abstract
Can a language model learn better ways to reason about other people's minds by improving its own strategies, and are the strategies it learns actually right? We study this with cogrsi. A grounded reader marks who saw each event in a story and uses a short written principle to settle disagreements between its readings. With a principle written by experts, it beats self-consistency at the same token cost. A self-improvement loop then writes the principle itself. It keeps an update only if accuracy rises by a calibrated margin, and in one variant only if it also passes counterfactual test pairs. The learned principles transfer: on 800 BigToM pairs that no system had seen, each of them scores higher than the expert principle. We then check what the learned rules actually say. Each rule is translated into code and run on thousands of small stories with known answers. Most of them give a wrong answer in at least one situation they claim to cover, for example by treating a character who is in the room as knowing what happened even while asleep. Accuracy does not reveal which rules are wrong, and the counterfactual test does not fix this. A strategy can help in practice and still be wrong as a rule, so both need checking. Our code and data are publicly available.