Neuro symbolic system improves learning Sudoku rules from few examples
Think-Verify-Revise: Neuro-Symbolic Visual Reasoning with Vision-Language Models and Dynamic Logic Tensor Networks
Computer Vision and Pattern Recognition
Summary
Figuring out rules from pictures is hard for computers when they only use brain-like patterns or strict logic by themselves. This paper shows a method that combines both approaches: a language-based model suggests logical rules after looking at just a few images, then a logic-based system checks if those rules work with the picture data. If the rules don't fit, the system tries again. This back-and-forth helps the computer learn Sudoku rules from only three example puzzles and does as well or better than earlier methods.
Neuro-symbolic learningVision-language modelsFirst-order logicDynamic Logic Tensor NetworkVisual reasoningRule inductionSudoku constraint rulesMachine learningConvolutional neural networks
Authors
Homayoun Afshari, Pietro Basci, Alessandro Russo, Lia Morra
Abstract
Visual reasoning tasks require a system to jointly perceive visual content and apply formal relational constraints---a combination that neither pure neural nor purely symbolic approaches handle well in isolation. This paper proposes a Neuro-Symbolic (NeSy) framework that closes this gap by tightly coupling a Vision-Language Model (VLM) for automatic First-Order Logic (FOL) rule induction with a Dynamic Logic Tensor Network (D-LTN) for differentiable rule verification, in a closed iterative feedback loop. The VLM receives a small set of labelled visual examples and proposes candidate FOL rules conforming to a strict grammar (Think); the D-LTN is automatically assembled from these rules at runtime and evaluates them grounding on CNN-produced visual embeddings (Verify); and verification failures are fed back to guide the VLM's next hypothesis (Revise). Evaluated on the ViSudo-PC benchmark across four visual domains (MNIST, EMNIST, KMNIST, FMNIST), the system induces valid Sudoku constraint rules using only three training examples as visual context. The proposed method achieves AUC scores matching or outperforming previous methods (NeuPSL, LTN), showing the potential for automatic rule discovery through VLM. Code is available at https://github.com/homayoun-afshari/nesy.