Papers for

offline rl platform operators

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Offline reinforcement learning faces hidden backdoor risks after data removal

UBA-ORL: Unlearning-Activated Backdoor Attacks on Offline Reinforcement Learning

Abstract: Offline reinforcement learning (offline RL) enables policy learning from pre-collected static datasets without online exploration, and is increasingly deployed not only in safety-critical domains such as autonomous driving and robotic control but also in data-mining applications such as recommendation and behavior analysis. While compliance-driven data removal enhances privacy, it also opens a previously unrecognized attack surface. We introduce UBA-ORL (Unlearning-activated Backdoor Attack on Offline Reinforcement Learning), the first unlearning-activated backdoor attack for offline RL: in the evaluated settings, the attack is substantially suppressed after normal training and becomes pronounced after a compliance-driven deletion (unlearning) request. UBA-ORL employs a dual-sample mechanism: alongside backdoor trajectories (BD) that link a trigger to malicious actions under inflated rewards, the attacker injects camouflage trajectories (CM) sharing the same trigger pattern but preserving benign actions with equally high rewards. During training, BD and CM provide competing supervisory signals; upon a legitimate deletion request on the CM subset, the residual BD signal can re-dominate, reactivating the backdoor on demand. Empirical results show that UBA-ORL achieves controllable activation under the evaluated offline-RL configurations, while no-trigger return changes vary by configuration, exposing a previously overlooked security risk in compliance-driven offline RL platforms. We urge the community to develop joint pre-/post-unlearning auditing mechanisms for compliant unlearning services.

Sat 19 SeptCryptography and Security
The gist
Offline reinforcement learning (RL) trains computer systems to make decisions using only existing data, without trying things out in real life. The authors found a sneaky way to hide harmful triggers in the training data that don’t show up during normal use but come back after specific parts of the data are deleted to follow privacy rules. Their method uses two types of data with the same trigger but different behaviors, so removing one type accidentally activates the harmful behavior. This reveals a new security risk in how offline RL systems handle data removal, especially as these systems are used for important tasks like driving or recommendations. The authors suggest creating new ways to check data before and after removal to keep these systems safe.
Open → 2609.22711v1