EML-type operators expand elementary functions in neural-style networks
Diversity of EML-type operators
Symbolic ComputationMachine Learning
Summary
The problem is finding simple mathematical tools that can calculate complex functions like exponentials and logarithms within a framework similar to neural networks. The authors identified many variations of a key tool called the EML operator that can do this better than previously known. They introduced a new neural network layer using rational functions and a special activation function that can separately calculate exponential and logarithmic functions. This helps in evaluating all elementary functions in a new way similar to how neural networks operate.
What this means in practice
- •For machine learning engineers: Construct neural network architectures that use rational functions to better model mathematical functions requiring exponentials and logarithms.
- •For scientific programmers: Implement symbolic regression methods within neural-style models to recover elementary transcendental functions efficiently.
Authors
Andrzej Odrzywołek
Abstract
The discovery of the EML operator, sufficient to evaluate the standard explicit purely transcendental elementary functions, has led to considerable interest and discussion across multiple scientific disciplines. However, most authors have focused on the binary EML itself, while numerous similar variants with slightly different properties are now known. This article attempts to close this gap by enumerating and classifying them. We also take this opportunity to clarify common misconceptions related to the EML operator. The principal goal, symbolic regression within an architecture as close as possible to proven neural networks which combine matrix multiplication with a single univariate non-linear activation function, remains beyond reach. Instead, we propose a Möbius layer, with rational functions replacing matrix operations, and showcase the recently discovered activation function eml(x,1/x), which allows exp(x) and ln(x) to be recovered separately, and hence all elementary functions to be evaluated within a rational generalization of the neural network.