Papers for

autonomous system operators

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Interpretable learned rules improve agent intervention reliability

Learn then Bet: Learning Interpretable Rules for Reliable Agent Intervention

Abstract: Autonomous agents now run for hundreds of steps with little or no human supervision. In practice, practitioners use runtime monitors such as turn limits and keyword filters to decide when a run should be stopped, upgraded, or assisted. However, these heuristics may stop runs that would otherwise succeed or allow failing runs to continue. To address this limitation, we introduce LeBet (Learn then Bet), a statistical learning framework that derives its learning objective from sufficient conditions for valid and powerful intervention. In particular, LeBet learns an interpretable OR-of-AND monitor and accumulates evidence through a fixed betting statistic, intervening when the evidence reaches a prescribed threshold. Under finite within-class exchangeability and leave-one-out stability, we derive finite-sample guarantees for false-positive control (validity) and detection power conditional on the training data, using the same trajectories for learning and calibration without assuming independent and identically distributed trajectories. To address the intractable combinatorial optimization problem in our learning formulation, we use stochastic local search that leverages the monitor's Boolean structure to propose rule edits aimed at improving empirical separation. On SWE-bench Lite with Ministral 3 8B, the learned monitor saves $34.46\%$ of input-plus-output tokens and increases output precision from $18.33\%$ to $21.59\%$. The empirical false-positive rate is $13.64\%$. Our results show how to formalize the learning problem over interpretable rule sets with statistical conditions and establish training-conditional guarantees for reliable intervention based on partial histories.

Fri 13 NovArtificial IntelligenceMachine Learning
The gist
When autonomous agents run without much human help, simple rules often decide when to stop or assist them, but these rules can make mistakes. The authors introduce a method called LeBet that learns easy-to-understand rules from data to decide when to intervene, doing so only when there is strong evidence. Their approach comes with mathematical guarantees that limit false alarms and improve detection based on how the system behaved in training. Tests with a large language model showed better precision and saved communication costs compared to simple rules.
Open → 2611.17662v1

Runtime auditor improves failure prediction in multi-agent systems

Dream2Discern: Learning from Hard Latent Variants for Runtime Failure Prediction in Multi-Agent Systems

Abstract: Reliable deployment of LLM-based multi-agent systems calls for runtime auditors that predict terminal outcomes from observed execution prefixes. These auditors typically learn from a limited corpus of trajectories with verified outcomes. Expanding this corpus requires costly execution and verification, yet broader failure coverage alone provides no explicit criterion for identifying prefixes that the current auditor finds difficult to classify. To this end, we propose Dream2Discern, a runtime auditor trained through implicit augmentation guided by its own prediction loss. During Dream, we select perturbations of the auditor's representation operators to increase prediction loss on recorded prefixes, yielding hard latent variants. During Discern, the auditor learns to predict execution outcomes from these variants using the original labels. The variants are regenerated as the auditor learns, turning a limited corpus into a renewable source of hard examples and requiring no additional trajectory collection or outcome verification. Extensive evaluations on AFTraj-2K and StepShield show that Dream2Discern outperforms competitive failure auditors and achieves over 90% trajectory-level weighted accuracy on both benchmarks while balancing prediction accuracy and timeliness. Project page: https://zbox1005.github.io/dream2discern-project/

Mon 12 OctMachine LearningArtificial IntelligenceComputation and Language
The gist
Predicting when complex groups of AI agents will fail during their tasks is important but difficult because verifying failures is costly and limited by available data. The authors propose a new training method called Dream2Discern, where the system generates challenging new training examples from existing data by tweaking its internal reasoning steps to increase prediction errors. This creates a renewable set of hard cases that help the system learn better failure prediction without collecting more data. Tests show this method improves accuracy and speed in predicting failures across benchmark datasets.
Open → 2610.16071v1

Safe delegation of AI actions through collective review and voting rules

Delegating Authorization to Misaligned Agents: Coalitional Alignment and Safe Control

Abstract: Long-running AI agents create a control problem: each action they take changes the state, which in turn affects the trajectory of future actions. If the agent is not fully aligned, then guaranteeing safety requires approving consequential actions before allowing them to be executed. But requiring human approval at every step makes attention a bottleneck. Delegating review to other AI agents raises the same alignment problem: the reviewers may themselves be misaligned. We identify a condition on a reviewing panel that is weaker than individual alignment yet necessary and sufficient for a guarantee that the principal fares at least as well in expectation as under a designated baseline policy. Each reviewer agent reports whether an action proposal made by a proposer agent improves its own utility relative to the baseline. We show that a threshold rule tolerating $k$ disapprovals is safe exactly when, after any $k$ reviewers are removed, the principal's utility can be written as a nonnegative combination of the remaining reviewers' utilities, plus a term that is nonnegative on every feasible proposal. We call this property $k$-robust coalitional alignment. The characterization lifts to sequential control: in a discounted MDP with an arbitrary proposer agent, safety at every state is both necessary and sufficient for the induced policy to match or improve on the baseline. When reviewers vote strategically, full-panel coverage in reward-function space guarantees that every Nash equilibrium is safe under the unanimous approval rule; in contrast, more permissive thresholds can admit unsafe equilibria even when reviewers are individually aligned. Experiments with existing reviewer models show that collective review can remain sound without an aligned individual, even when some disapprovals are tolerated.

Mon 14 SeptComputer Science and Game TheoryArtificial IntelligenceMachine Learning
The gist
When AI systems make decisions over time, it’s hard to keep them safe if they don’t perfectly follow human goals. Having humans approve every action is too slow, so the authors study letting other AI agents review decisions. They find a way to ensure overall safety by requiring a group of reviewers to agree in a specific way, even if some reviewers disagree or are imperfect. Their method guarantees the AI system’s decisions won’t be worse than a baseline and works even when reviewers try to act strategically.
Open → 2609.15803v1