Remote monitor strategies minimize incorrect info age in pull systems
Minimizing Age of Incorrect Information in Pull-Based Remote Estimation of a Markov Source
Information TheoryNetworking and Internet Architecture
Summary
The paper looks at how a remote monitor can ask for updates from a changing information source to keep its data accurate. The authors focus on situations where updates take either no time or one time unit to arrive through a channel. They develop mathematical ways to decide the best times for the monitor to ask for updates, aiming to reduce how long and how often it holds wrong information. For the more complex one-unit delay case, they use advanced computation methods, including deep reinforcement learning, to find these best strategies.
What this means in practice
- •For network schedulers: Optimize the timing of data pull requests to keep network data fresh and accurate under delay and sampling constraints.
- •For industrial control teams: Improve remote monitoring systems for processes with changing states by minimizing outdated or incorrect status information.
Authors
Ismail Cosandal, Sennur Ulukus, Nail Akar
Abstract
In this paper, we consider a discrete-time pull-based remote estimation system involving an information source and a remote monitor where the information source is a finite-state irreducible discrete-time Markov chain (DTMC) process. In this system, the remote monitor sends a pull request towards the source which responds by sampling its process so as to transmit the sampled values towards the monitor through a delay channel, encapsulated in information packets. We consider i) zero delay (ZD) and ii) unit delay (UD) channels, where the packet transmission times are zero and one slot, respectively. For remote estimation, we use the maximum a posteriori (MAP) estimator, and for quantifying its performance, we use the mean age of incorrect information (MAoII) which is a recently proposed freshness and mismatch metric that penalizes an incorrect estimation along with its duration. Our goal is to obtain pull policies so as to minimize MAoII under a long-run sampling rate constraint. For the ZD model, we propose a semi-Markov decision process (SMDP) formulation of the problem, which allows us to find the optimum observation-dependent pull policy. On the other hand, the UD channel prevents the monitor from possessing perfect information on the actual AoII process at any given time, and the SMDP formulation of the ZD channel cannot readily be used for the UD model. For the system with the UD channel, we first provide a numerical algorithm to calculate the MAoII and the average sampling rate for a given observation-dependent pull policy. For the optimization of the underlying remote estimation system, we use the joint distribution of the age and source processes to be obtained from the history of all observations, termed as belief. Subsequently, we propose a belief-dependent policy obtained via the belief-MDP formulation and a constrained deep reinforcement learning (CDRL) algorithm.