Task aware approach improves decisions in logic gate network discretization
Task-Aware Discretization of Differentiable Logic Gate Networks
Machine LearningLogic in Computer Science
Summary
Many artificial intelligence models use logic gates to make decisions, and training these models often involves smoothing out these gates to allow gradual learning. The authors found that deciding when and how to make these gates fully discrete (finalizing their behavior) based only on local confidence can lead to worse overall performance. They developed a new way to include information about the whole task when making these decisions, improving how the model freezes its logic gates. Their experiments with image-related neural networks showed that considering nearby gates one at a time works well, but trying to optimize many gates together is less reliable.
What this means in practice
- •For machine learning engineers: Optimize discretization steps in Boolean network models to improve efficient inference accuracy in deployed systems.
- •For hardware accelerator designers: Improve training-to-inference translation of logic gate networks for more reliable hardware implementations using task-aware discretization.
Authors
Thore Gerlach
Abstract
Differentiable logic gate networks (DLGNs) enable gradient-based training of highly efficient Boolean networks by relaxing discrete logic gates during training and discretizing them for inference. Standard approaches make this discretization decision locally, typically through argmax selection and confidence- or entropy-based convergence criteria. We show that local discretization can be task-suboptimal even for globally optimal relaxed solutions, with high gate confidence providing no general guarantee, and derive bounds relating task-aware gate selection to tractable interventions in the relaxed network. Motivated by these results, we study first-order downstream task information for progressive discretization and characterize when this local approximation is reliable. Experiments on convolutional DLGNs reveal a strong locality dependence: first-order scores become unreliable when directly optimized over nonlocal interventions, but accurately assess local argmax decisions for progressive freezing.