Neuro-symbolic models enable real-time human-guided AI decisions
Generative Interpretability via Scalable Neuro-Symbolic Models
Machine LearningArtificial IntelligenceComputation and LanguageSymbolic Computation
Summary
Current AI explanations happen after decisions are made, which limits safety when AI actions affect the real world. The authors suggest redesigning AI so that it reveals understandable steps during its decision-making, not just after. Their approach uses neuro-symbolic models, which combine neural networks with symbolic reasoning, to create checkpoints humans can understand and modify before final decisions are taken. This method could make AI systems safer and more trustworthy by allowing interventions in real time.
What this means in practice
- •For ai system developers: Build AI agents that expose understandable decision steps during operation to allow intervention before actions are finalized.
- •For autonomous vehicle teams: Implement models that reveal interpretable checkpoints to audit and correct decisions before irreversible driving actions.
A position paper. It proposes an approach and reports no results.
Authors
Xiaocong Yang
Abstract
As the use of Large Language Models moves from chatbots into agentic systems, where outputs become actions with irreversible consequences on reality, the existing paradigm on AI Interpretability research, post-hoc interpretability, is structurally inadequate for safe and trustworthy model deployment: it explains behavior after the fact but cannot audit or intervene in an inference computation before it commits to an output. We therefore argue for a shift toward \emph{generative interpretability}, an architectural property under which a model's inference pass natively exposes semantically meaningful checkpoints that are human-understandable and amenable to causal intervention. We show the merits of generative interpretability as comparison to other interpretability research paradigms, and propose Neuro-Symbolic Models as a concrete instantiation.