On the Entropy Formula for Real, Complex, and Quaternionic Deep Linear Networks
2026-06-15 • Machine Learning
Machine Learning
AI summaryⓘ
The authors build on a previous formula about entropy, which is a way to measure uncertainty or information, in deep linear networks (DLNs) that use real numbers. They expand this formula to also work for DLNs using complex numbers and quaternions, which are extensions of real numbers with more dimensions. This creates one combined formula that applies to networks with real, complex, or quaternion inputs. Their work helps unify how entropy is understood across these different types of number systems within DLNs.
EntropyDeep Linear NetworkReal numbersComplex numbersQuaternionsUnified formulaInformation theoryMathematical extension
Authors
Luis Contreras, Marco Nahas, Tejas Kotwal
Abstract
We extend the entropy formula of Menon and Yu for the real Deep Linear Network (DLN) to its complex and quaternionic analogues, obtaining a unified formula for DLNs over $\mathbb{R}$, $\mathbb{C}$, and $\mathbb{H}$.