Active feature acquisition methods adapt to missing training data costs and performance
Active Feature Acquisition With Incomplete Training Data
Machine LearningArtificial Intelligence
Summary
Sometimes gathering all information needed to make a prediction is too costly or impossible. The authors study how to actively decide which pieces of information (features) to get, even when the available training data is incomplete or missing some parts. They find that certain approaches to handle missing data work better than others, especially when planning multiple steps ahead. Their experiments show that methods using a generative model to restore missing data can recover much of the lost performance. This helps build smarter systems that gather information efficiently despite incomplete data.
What this means in practice
- •For medical data analysts: Design diagnostic tools that selectively request patient tests while handling incomplete historical medical records effectively.
- •For automated sensor networks: Optimize sensor activations in environments with partial or missing training data to reduce cost without losing accuracy in monitoring.
Authors
Reza Rezvan, Valter Schütz, Han Wu, Linus Aronsson, Morteza Haghir Chehreghani
Abstract
In many prediction tasks, acquiring all features can be a prohibitively expensive or outright impossible task. Further, in many cases a static subset of features may not be enough to solve the problem sufficiently across various instances. Active Feature Acquisition (AFA) addresses these problems by formalizing the trade-off between feature cost and predictive performance during sequential feature selection. However, prior AFA work largely assumes access to complete training data, an assumption that is often violated in practice. Here we study AFA with Incomplete Training Data (AFA-ITD), showing that under missing completely at random (MCAR) data, one-step acquisition values remain unchanged, whereas multi-step values can decrease. We analyze three approaches to learning from incomplete data: aliasing, filtering, and generative restoration. We show that filtering can require a number of training instances scaling exponentially with the dimension, whereas generative restoration scales exponentially with the acquisition budget. We empirically test our theory on a controlled experiment and across common AFA datasets and find that missingness mainly damages methods that exploit multi-step acquisitions and that generative restoration is able to recover lost performance in many experiments. Code is available at https://github.com/Linusaronsson/AFA-Benchmark/tree/missing-data.