Task scope shapes information needs more than receiver type
On Task Scope and Information Retention in Source Coding
Information TheoryMachine Learning
Summary
Deciding what information to keep or throw away when compressing data depends on the task you want the data to help with, not just who will receive it. The paper shows that whether a computer or a human uses the data doesn’t alone say how much detail must be saved. Instead, the kinds of predictions and decisions the data supports, and their accuracy requirements, really determine how much information is needed. The authors explain this by extending classic data compression ideas to handle multiple tasks and their demands.
What this means in practice
- •For data compression engineers: Design compression algorithms that adapt data retention based on specific prediction tasks instead of just whether output is for machines or humans.
- •For machine learning system builders: Optimize how sensor or feature data is encoded to support multiple automated tasks with differing accuracy and risk requirements.
A theory result. No direct application yet.
Authors
Alireza Furutanpey, Kerstin Bunte
Abstract
We argue that dividing codec design into Coding for Machines (CfM) and Coding for Humans (CfH) is a misleading distinction for deciding what information a codec may discard. Receiver identity does not determine admissible information loss. The required rate depends on task scope, including the predictions to support, their losses and tolerated risks, the encoder observation, and the permitted decoding procedures. Notably, a machine task may have a higher minimum rate than a restricted human decision. Rate savings on selected machine tasks apply only to the stated requirements, not to an intrinsic ordering by receiver type. We extend source and feature coding to finite task families, derive when restricting the encoder observation preserves the minimum rate, and show that equality between source and split-feature coding rates can no longer hold as the task scope expands.