Summary
Scientists want to understand parts of a process they can’t directly observe, like how water leaves plants versus the ground. The authors look at models that combine real-world knowledge with neural networks to estimate these hidden parts from overall measurements. They found that simply training these models can miss many different possible hidden explanations that still fit the observed data. To fix this, the authors developed a special way to train the models so they explore more of the possible hidden patterns without messing up the fit to what we see. Their method carefully balances learning from main data and exploring alternatives, improving understanding of hidden processes.
What this means in practice
- •For environmental modelers: Produce more diverse and accurate estimates of unobserved water flux components while preserving fit to observed data.
- •For machine learning engineers: Improve multi-objective neural network training by preventing interference between main loss and auxiliary gradients under AdamW optimization.
Authors
Ekaterina Gikalo, Zavud Baghirov, Christian Reimers, Markus Reichstein, Martin Jung, Marco Körner
Abstract
Hybrid models let neural networks supply the unresolved parameters of a process-based simulator, and are fitted to observations that constrain aggregate fluxes and storages but not the internal partitioning that produces them. Such a fit is equifinal: configurations indistinguishable in the observed aggregates can disagree substantially about the unobserved internals, and therefore about the mechanism they attribute to the same data. We study evapotranspiration, whose total flux is observed while its split into transpiration and evaporation is not. A model ensemble built from random seeds alone spans only a narrow band of the partitionings the data admit. We produce the spread deliberately, training members under auxiliary repulsion on transpiration, subject to the fit to observations not degrading. Non-interference is usually arranged by making the auxiliary gradient orthogonal to the main one, but under AdamW the applied step is a preconditioned momentum direction, so orthogonality between raw gradients no longer implies it. We introduce Non-interfering Asymmetric Network Optimization (NANO), an update whose first-order effect on the main loss is exactly that of the main objective alone, whose auxiliary step is bounded, and which concentrates repulsion in the coordinates the main gradient leaves idle.