Papers for

financial algorithm developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Adaptive design improves agent behavior in complex environments

Adaptive Agent Design

Abstract: We consider an agent acting against a general non-Markovian environment. The agent maintains its agent states, but is free to choose a transition kernel across those states and optimize its state-feedback control policies. We study the bi-level agent design problem that optimizes the transition kernel and the policy it induces, given said kernel with offline data of observations and actions obtained via a behavioral policy. For general environments, we show that a soft $Q$-learning algorithm converges almost surely to the fixed point of a soft Bellman equation defined by the stationary averages that the behavioral policy and the chosen kernel induce, and we delineate what separates the resulting policy from an optimal one. In partially observed Markov decision problems, we analyze convergence properties of parametrized transition kernel design via zero-th order and Bayesian optimization techniques.

Fri 11 SeptArtificial IntelligenceComputer Science and Game Theory
The gist
Some computer programs called agents have to make decisions based on past actions and observations, but the situations they face can be complicated and not follow simple rules. The authors studied how an agent can learn both how it moves between internal states and how it acts to get the best results using existing data. They showed that a learning method called soft Q-learning can find good solutions even when the environment does not follow simple assumptions, and they explored ways to improve how the agent changes its internal state transitions in partly observable settings. This work helps understand how to create smarter agents that adapt to complex and uncertain situations.
Open → 2609.12486v1