Continuous-time control finds boundary between learning and no-learning phases
Sharp Critical Minimax Laws and No-Learning Thresholds in Continuous-Time Adaptive Control
Machine Learning
Summary
This paper studies how to control a system continuously over time when some details of the system are unknown. The authors focus on how well it's possible to adapt control strategies to improve performance compared to doing nothing, especially when the unknown part is small. They identify a precise mathematical rule that describes when learning to improve control is beneficial and when it is not. This helps understand the fundamental limits of adaptive control in situations with uncertainty.
What this means in practice
- •For control engineers: Determine when adaptive control can meaningfully reduce costs versus no control by using the precise minimax bounds and phase transition results from this paper.
- •For robotics system designers: Use the no-learning thresholds to design adaptive controllers that avoid unnecessary complexity when system parameters fall below a critical threshold.
A theory result. No direct application yet.
Authors
Chen Jia
Abstract
We study episodic continuous-time control with an unknown vector control gain, scalar state, quadratic action cost, and smooth convex terminal cost. In the scalar Gaussian experiment, let $Δ(H)$ denote the minimax improvement over zero control and set $δ=\sqrt2H^2-1$. We prove the critical law $$ Δ(H)\asymp δ^4\sqrt{\log(1/δ)} \qquad (δ\downarrow0), $$ with a matching lower and upper bound. The lower bound follows from a uniform deficit--energy inequality valid for fully adaptive controls with unbounded amplitudes, while a moving soft-threshold feedback attains the rate. For local parameters $θ=N^{-1/4}h$, $|h|\le H$, we show that the normalized minimax regret over $N$ episodes is within $O(N^{-1/2})$ of a fixed-horizon Gaussian sequential control problem, without an additional dimension factor. The terminal task enters the limit only through $c_g=\operatorname{Var}(g(Z))$. The Gaussian problem exhibits an exact no-learning phase transition: $$ C_d^T(H)=TH^2/2 \iff H^4T\le d^2/2, $$ yielding the asymptotic task boundary $c_gH^4=d^2/2$.