Papers for

online recommendation engineers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

New algorithm balances safety and learning in sequential decisions

Three Worlds for the Price of a Logarithm: Baseline-Protected Best-of-Both-Worlds Combinatorial Semi-Bandits

Abstract: Improving on a baseline requires exploring away from it, precisely where its performance may be lost. We propose DUET (Doubling Under Empirical Tests), a horizon-free phased mixing scheme for combinatorial semi-bandits with arbitrary finite action sets. Two observable tests decide when to double the learning weight and when to double the empirical-regret budget and restart. We instantiate the scheme with entropy-FTRL in mixed phases and hybrid-FTRL in pure phases. Assuming the baseline selects every arm with positive probability, this instantiation keeps expected baseline regret logarithmic in $T$, while its expected regret to every fixed action is $\widetilde{O}(\sqrt{T})$ under non-anticipating adversarial losses and polylogarithmic, gap-dependent, under i.i.d. losses with a unique optimal action. All three guarantees hold at every deterministic horizon, and the algorithm knows neither the horizon, nor the gap, nor the environment type. Baseline protection, adversarial robustness, and stochastic adaptivity thus come from one scheme and two base learners, at the price of a logarithm.

Mon 26 OctMachine LearningComputer Science and Game Theory
The gist
Making better choices than a known baseline can be risky because it involves trying new options that might perform worse. The authors propose a new method called DUET that smartly decides when to explore and when to rely on the baseline to keep performance safe. This method adapts to different environments without knowing ahead of time how long it will run or how good the best option is. Their approach maintains strong safety guarantees while learning effectively in both adversarial and normal, random settings.
Open → 2610.36245v1

Large language model plugins risk covert hijacking for profit

Steering for Safety, Hijacking for Profit: Trojan Threats in the LLM Activation Steering Supply Chain

Abstract: As large language models (LLMs) are increasingly integrated into Web search, online recommendation, and intelligent decision-making, the third-party plugin ecosystem centered on model customization continues to grow. Among these, Steering Vectors guide model behavior by intervening in internal representations. However, the open plugin ecosystem also introduces new supply-chain security risks: malicious plugins can disguise themselves as having legitimate functionality and covertly manipulate outputs without significantly affecting model performance. Although this risk poses a real threat, the security issues of Steering plugins in the LLM supply-chain ecosystem still lack systematic research. This paper audits eight mainstream open-source Steering plugin libraries and reveals three structural vulnerabilities: insufficient distribution integrity verification, excessive plugin privileges, and missing runtime monitoring, which offers an attack surface for attackers. To investigate this threat, we propose a malicious plugin, IDH. Specifically, IDH designs a norm-gated mutually exclusive routing mechanism and proposes an information-decay inflection point localization method to automatically select the injection layer, thereby simultaneously achieving benign and malicious functions. To counter such threats, we propose a runtime defense framework, by suppressing anomalous activation interventions and orthogonalizing inter-layer trajectory. We evaluate IDH across five mainstream LLMs, using safety-refusal steering as the legitimate functionality and commercial recommendation manipulation as the hijack objective. Experiments covering six brands across three commercial domains demonstrate that IDH achieves an attack success rate (ASR) of up to 97.8% while preserving high refusal rates on harmful requests.

Mon 26 OctCryptography and Security
The gist
The paper shows that plugins designed to steer large language models (LLMs) can be secretly hijacked by attackers to manipulate outputs for profit without obvious signs. The authors studied eight popular plugin libraries and found key weaknesses that let malicious code sneak in and take control during model use. They created a proof-of-concept malicious plugin that can both behave normally and perform harmful actions, demonstrating a high success rate across various LLMs. To combat this, they propose a defense system that detects and limits suspicious plugin activity at runtime to keep the models safer.
Open → 2610.35043v1

Online treatment selection controls errors while learning from limited feedback

Online Individualized Treatment Selection under Partial Feedback: False Discovery Control and Minimax Optimality

Abstract: We study online individualized treatment selection when only the outcome under the assigned treatment is observed. Our objective is to preferentially treat individuals whose potential outcomes under treatment exceed a covariate-specific control reference, while limiting false discoveries among treated individuals. We propose Partial-Feedback Adaptive Individualized Treatment Selection (PAITS), which initializes learning from historical data and combines sequential estimation of local null risks, randomized exploration, and feedback-driven updates of the treatment selection threshold. Under regularity conditions, we establish asymptotic false discovery rate control and sublinear gap-weighted assignment regret relative to a population oracle. We further establish a minimax lower bound on assignment regret under prescribed uniform nuisance-estimation requirements and give conditions under which PAITS attains the corresponding constrained minimax rate up to logarithmic factors. These results quantify the tradeoff between accurate treatment assignment and nuisance learning under partial feedback. Simulation studies and a real-data analysis using the Jester recommendation dataset illustrate the finite-sample performance of PAITS.

Mon 12 OctMachine Learning
The gist
When deciding who should get a treatment based on their personal characteristics, we often only see the result for those who actually get the treatment, but not what would have happened otherwise. This paper studies a method called PAITS that learns over time to pick treatments for individuals while controlling mistakes where the treatment is not actually better. The method starts with existing data, tries some options randomly, and updates how it chooses treatments based on new results. The authors prove that PAITS balances learning well and making few errors, and show it works using simulations and one real dataset.
Open → 2610.16464v1

Attack methods boost target choices in bandit learning with low cost

Beyond Reward Suppression: Near-Optimal Offline Attacks on Warm-Start Bandits with Bounded Rewards

Abstract: Adversarial attacks on bandits aim to mislead a learner toward a target arm while keeping the attack cost small. Existing attacks typically achieve this by suppressing non-target arms. In practice, however, manipulation such as fake reviews often directly promotes the target item. We study this gap through bounded offline attacks on warm-start bandits, where an attacker can inject only valid action-reward pairs into the warm-start history before deployment. We show that target promotion is not merely a heuristic: when the target arm lies near the lower reward boundary, any order-optimal-cost attack against UCB that makes it selected in nearly all online rounds must allocate a nonvanishing fraction of its cost to the target arm. We then design an attack that achieves the optimal sublinear cost and characterize its allocation between target promotion and non-target suppression. We further extend the attack to Thompson Sampling, $ε$-greedy, and a broader class of bandit algorithms. Experiments on real-world and synthetic data validate the effectiveness of our attacks.

Wed 7 OctMachine LearningArtificial Intelligence
The gist
Bandit learning algorithms decide what choices to make based on rewards they receive, but sometimes attackers try to trick them to pick certain options more often. The paper studies attacks done before the algorithm starts, by adding fake past data to guide the learning. The researchers found that attacking by promoting the target choice, not just hurting others, is important and efficient in some cases. They created attacks that balance promoting the target and suppressing others, working for several common learning strategies. Their tests on real and fake data show these attacks can effectively mislead bandit algorithms at a small cost.
Open → 2610.10000v1

LinUCB learns efficiently by sampling like optimal experimental designs

Sampling Allocation of LinUCB: Optimal Design Limits in the Small-Gap Regime

Abstract: We study the sampling allocation of LinUCB in the small-gap regime, where the reward gaps are of order at most $n^{-1/2}$ over the decision horizon $n$. This scaling captures the hard instances underlying worst-case regret lower bounds, for which LinUCB is known to be near optimal up to logarithmic factors in $n$. Using a mean-field perspective, we characterize this allocation through the empirical sampling distribution, a macroscopic object that averages the effect of adaptive decisions over the horizon, and identify its limit as $n\to\infty$. We establish that in this regime, the empirical sampling distribution induced by LinUCB converges to the set of D-optimal designs. This central result reveals that, in the small-gap regime, LinUCB not only achieves near optimal minimax regret but also allocates samples in a way that is asymptotically efficient for learning the reward parameter, thereby connecting regret-driven online learning with information-efficient experimental design. Building on the optimal design limit, we obtain two useful consequences. First, we refine the asymptotic regret analysis of LinUCB in the small-gap regime by characterizing its leading-order constant in the limit. Second, we show that, despite LinUCB's adaptive sampling strategy, the regularized least-squares estimator satisfies a central-limit-type theorem in the small-gap regime, thereby enabling valid statistical inference for the reward parameter.

Mon 5 OctMachine Learning
The gist
Some algorithms decide which option to try next by learning from past choices. When the differences between options' rewards are very small, understanding how samples are allocated can help improve performance. The authors show that LinUCB, a popular decision algorithm, samples in a way that matches the best possible experiment designs when many choices are made. This means LinUCB not only minimizes losses but also gathers information efficiently to learn the true rewards. Their work also helps in making precise guesses about reward values and understanding how reliable those guesses are.
Open → 2610.06213v1

New method learns optimal choices in two-arm bandit problems faster

Elicitation and Decision Geometry in Single-Index Bandits

Abstract: We study two-arm contextual bandits with arm-specific single indices and a shared unknown monotone link. Monotonicity makes the optimal action depend only on the contrast between the index directions, hence arm-specific reward functions need not be estimated. We introduce Natural Boundary Learning (NBL), a greedy procedure that uses a sequential Stein contrast to learn the optimal boundary directly, without estimating the reward functions or the common link. We characterize the local Riemannian dynamics of NBL through a decision stability coefficient balancing arm separation, link geometry, and the context distribution. We show that this stability is connected to the elicitation geometry of the underlying convex potential. Under local decision stability, NBL contracts toward the optimal boundary and achieves $O(\log n)$ expected regret. Numerical experiments illustrate the predicted stability regimes and compare NBL with a parametric greedy benchmark under link misspecification.

Mon 28 SeptMachine Learning
The gist
This paper deals with a decision-making problem where you have two options, each with unknown rewards that depend on context. The authors develop a new method called Natural Boundary Learning (NBL) that learns the best choice directly without needing to estimate how each option behaves separately. They show that this approach quickly improves decisions over time by focusing on the boundary between the two options. Their analysis explains when and why the method works well, and experiments confirm these findings.
Open → 2609.35622v1

Error limits found for estimating preferences with the BTL model

Error Bounds for Statistical Estimators in BTL Model with Parametric Multivariate Utility Functions

Abstract: We study preference elicitation under the Bradley-Terry-Luce (BTL) model where the true partworth vector is unknown and has to be estimated as a parameter with elicited preference information. The set of selected pairwise queries is non-uniform, deterministic, and arbitrary over a collection of alternatives, provided that it satisfies a joint identifiability condition. We focus on understanding when the canonical maximum likelihood estimator (MLE) is finite and admits sharp error bounds without explicit compactness constraints on the feasible set or external regularizers. To this end, we derive minimax lower bounds under the standard bounded dynamic range condition, and find that the same Fisher-information geometry in the classic Cramér-Rao lower bounds underpins the finite-sample difficulty of the estimation problem. By combining a non-asymptotic expansion of the likelihood score equation with a fixed-point localization argument, we identify a design-dependent sample size threshold above which the unconstrained canonical MLE exists and is unique with high probability. The same expansion yields a decomposition of the estimation error into a linear stochastic term, an explicit second-order bias, and a higher-order remainder. A refined analysis gives sufficient sample size conditions under which the canonical MLE attains the minimax rates up to logarithmic and constant factors. These results provide a unified non-asymptotic theory for parametric utility elicitation and reveal when the inference is determined by response data alone rather than by external regularization. Preliminary numerical results are consistent with the theoretical findings.

Tue 22 SeptMachine Learning
The gist
When trying to understand people's preferences by comparing pairs of choices, the true preferences are unknown and must be estimated from data. The authors studied conditions under which the usual method for estimating these preferences works well without extra constraints. They showed when this estimator is guaranteed to exist, be unique, and how accurate it is depending on the number and design of comparisons. Their findings give precise, non-asymptotic bounds on the estimation error and show when the data alone is enough for reliable inference. Initial tests confirmed their theoretical results.
Open → 2609.26326v1