Expressive learning method finds all inputs meeting specific targets
Representation Learning for Exact Preimages
Machine Learning
Summary
Many technologies predict outcomes from inputs, but sometimes we need to work backwards: find all inputs that lead to a certain result or safety level. This is challenging because complex predictors don't easily show these inputs or let us optimize over them. The authors propose TRIO, a technique that keeps powerful predictions while representing input sets geometrically, making it easier to find all inputs that meet goals. TRIO guarantees closely approximating any needed prediction and its full set of matching inputs, enabling exact and efficient backward reasoning.
What this means in practice
- •For safety engineering teams: Identify all operating conditions that guarantee equipment safety levels using expressive learned models with exact inverse input sets.
- •For control system developers: Design control inputs that exactly achieve performance targets by optimizing over explicitly represented sets of valid inputs.
Authors
Konstantin Hess, Stefan Feuerriegel
Abstract
Modern neural predictors can model highly nonlinear maps, but many scientific and engineering tasks require reasoning in the opposite direction: given a performance or safety level, the goal is to characterize the preimage, that is, the complete set of inputs which meet the desired target level and optimize over that set. For expressive neural predictors, however, such preimages typically have no explicit representation and are expensive to recover or optimize over. This creates a fundamental three-way challenge between expressive forward prediction, accurate preimage approximation, and tractable optimization over the preimage for downstream tasks. We introduce TRIO (tractable representations for preimage learning and inverse optimization), a framework for learning representations that make these objectives compatible by construction. Our key contribution is a preimage factorization: the forward model remains expressive through nonlinear radial transformations (including neural networks), while, under inversion, each transformation reduces to a single scalar radius, which yields simple geometric level sets. This yields an explicit geometric representation that is reusable for downstream optimization over the preimage, and, for linear objectives, we show that this admits a closed-form global solution. We finally prove a universal approximation theorem which shows that TRIO can approximate any continuous forward map and its entire family of potentially disconnected, nonconvex preimages arbitrarily well. Hence, TRIO combines expressive forward modeling, exact preimage recovery, and tractable global downstream optimization over preimages by design.