Logic gate networks overcome redundancy to improve accuracy and scale
Overcoming Kernel Redundancy for Scaling Logic Gate Networks
Computer Vision and Pattern Recognition
Summary
Logic gate networks are an alternative to neural networks that use only logic gates to do their calculations. The authors found that when these networks get wider (have more kernels), they often don’t get better because many kernels end up being redundant or similar. To fix this, the authors designed a way to keep kernels specialized and routed based on input, especially focusing on the first gate layer. This method uses only logic gates even at inference time and results in networks that use their kernels more effectively and achieve better accuracy.
What this means in practice
- •For hardware accelerator designers: Design logic gate-based accelerators with improved scalability by reducing kernel redundancy through dynamic routing at the gate level.
- •For embedded system engineers: Implement efficient logic gate networks with better accuracy using dynamic kernel specialization, suitable for resource-constrained devices.
Authors
Sejin Park, Hongjae Lee, Changwoo Han, Seung-Won Jung
Abstract
Differentiable logic gate networks, which operate using only logic gates, have recently attracted attention as an efficient alternative to conventional neural networks. However, despite their efficiency, the scaling behavior of logic gate networks remains underexplored. By contrast, scaling model capacity is a central design principle in deep neural networks and typically leads to improved performance. This discrepancy raises a key question: Can similar scaling benefits also be achieved in logic gate networks? In this work, we focus on width as a primary scaling axis and conduct a systematic analysis of its behavior in logic gate networks. We observe that naive width scaling often introduces redundancy among logic kernels, limiting the effective use of additional kernels and leading to performance saturation. To address this limitation, we propose a dynamic logic kernel framework that reorganizes kernel utilization by promoting specialization across kernel groups. This enables the network to better utilize increased width via input-dependent kernel routing, while ensuring that both routing and computation are implemented entirely with gate-level Boolean operations at inference time. We further find that kernel redundancy is most pronounced at the first gate level, motivating an early-stage dynamic logic kernel strategy that concentrates adaptation at this level. Experimental results demonstrate that our approach improves kernel utilization and increases kernel diversity, leading to higher accuracy with improved parameter efficiency.