Multiple Additive Neural Networks for Structured and Unstructured Data
2026-04-29 • Machine Learning
Machine Learning
AI summaryⓘ
The authors present a method called Multiple Additive Neural Networks (MANN), which improves on traditional Gradient Boosting by using simple neural networks instead of decision trees. They use special types of neural networks like Convolutional and Capsule Networks to handle different data types, including images and audio. The authors show that MANN can learn continuously, avoid overfitting, and works well without needing much fine-tuning. Their tests indicate MANN performs better than standard techniques like Extreme Gradient Boosting on common datasets.
Gradient BoostingNeural NetworksConvolutional Neural NetworksCapsule Neural NetworksFeature ExtractionOverfittingHyperparametersExtreme Gradient BoostingStructured DataUnstructured Data
Authors
Janis Mohr, Jörg Frochte
Abstract
This paper extends and explains the Multiple Additive Neural Networks (MANN) methodology, an enhancement to the traditional Gradient Boosting framework, utilizing nearly shallow neural networks instead of decision trees as base learners. This innovative approach leverages neural network architectures, notably Convolutional Neural Networks (CNNs) and Capsule Neural Networks, to extend its application to both structured data and unstructured data such as images and audio. For structured data the advantages of capsule neural networks as feature extractors are used and combined with MANN as a classifier. MANN's unique architecture promotes continuous learning and integrates advanced heuristics to combat overfitting, ensuring robustness and reducing sensitivity to hyperparameter settings like learning rate and iterations. Our empirical studies reveal that MANN surpasses traditional methods such as Extreme Gradient Boosting (XGB) in accuracy across well-known datasets. This research demonstrates MANN's superior precision and generalizability, making it a versatile tool for diverse data types and complex learning environments.