Symbolic models reveal shared simplifications in human and machine reasoning
Synthesizing symbolic models of human and machine reasoning at scale
Artificial IntelligenceMachine Learning
Summary
People and AI systems often try to explain complex problems using simpler, rule-based approaches. The authors developed a new method called Behavior-Conditioned Symbolic Synthesis (BCSS) that automatically creates clear symbolic programs to model reasoning behavior. By applying BCSS to hundreds of reasoning problems, they found that both humans and machines tend to oversimplify and sometimes misinterpret problem features. This shows that despite differences, humans and machines share similar reasoning shortcuts that depend on how much effort they put in.
What this means in practice
- •For ai system developers: Create interpretable symbolic models to understand and diagnose how AI systems make decisions on complex reasoning tasks.
- •For cognitive modelers: Automatically generate symbolic explanations of human reasoning behavior across diverse problem sets, improving insight into cognitive strategies.
Authors
Solim LeGris, Philipp Mondorf, Jinran Jin, Akshay K. Jagadish, Brenden M. Lake, Todd M. Gureckis
Abstract
Symbolic models offer a powerful account of cognition, but their scope has historically been limited by the need for hand-engineering and computational tractability. We show how advances in artificial intelligence (AI) can scale symbolic modeling and help overcome these engineering limitations. We introduce Behavior-Conditioned Symbolic Synthesis (BCSS)---an approach for automated search of interpretable, deterministic symbolic programs that verifiably reproduce behavior---and apply it to model both human and machine behavior across hundreds of challenging reasoning problems. Analyzing these programs reveals that both people and machines tend to oversimplify, adopt rules that are partial explanations for their observations, and misinfer dependencies between problem features. These shared signatures are modulated by available resources, with reasoning effort predicting the extent to which machines simplify and explain their observations. Together, these results point to underlying mechanisms reminiscent of satisficing in both people and machines, and illustrate how BCSS offers a general approach for interpreting and comparing rule-based reasoning of humans and black-box AI systems.