Lair Net improves predictions for complex tabular data patterns

LAIR-Net: Leaky Alignment-Impulse Residual Networks for Tabular Regression

Machine Learning

Summary

This work looks at how to improve making predictions from tables of data with many columns. The authors introduce LAIR Net, a special kind of network that mixes a learned reference input at each step to better keep track of patterns. They show that this helps when the relationship in the data is complex and not just a simple line, especially when there's reasonable noise in the data. They tested their approach on many benchmark datasets and found it outperforms many other methods on average.

What this means in practice

Authors

Rahul Goswami, Aryan Bhambu, Bittu Karmakar

Abstract

Deep randomized models fix hidden-layer parameters through random initialization and learn only closed-form readouts, typically adding depth by stacking random trans formations without target-aware control of hidden-state evolution. We propose LAIR Net, the Leaky Alignment-Impulse Residual Network, which mixes a shallow learned anchor into each hidden state through a leaky residual transition. We derive a depth uniform bound on input-perturbation sensitivity and use controlled simulations to attribute gains over a randomized baseline to the anchor rather than recursion or added capacity. Benefits emerge when a nonlinear target structure is learnable at the available noise level and diminish for nearly linear targets or dominant noise. Across 23 benchmark datasets, LAIR-Net achieves the best average rank among eight randomized networks and twelve conventional models, with relative performance associated with the same nonlinear-structure and noise quantities identified in simulation.