Summary
Generative recommendation systems, which suggest items like movies or products by predicting the next best choice, can be tricked by attackers who add small amounts of fake data. The authors found that injecting less than 1% of fake interactions can make these systems unfairly promote harmful or unwanted items when certain triggers appear. They studied why this happens and developed a method called AlignPoison that finds special starting points in the system’s predictions to make targeted attacks more effective. Their experiments showed that this approach significantly increases how often these systems promote the targeted items, revealing a new security weakness.
What this means in practice
- •For recommendation system engineers: Detect and mitigate poisoning attacks by understanding how small fake data injections cause targeted item promotion in generative recommenders.
- •For cybersecurity teams: Develop monitoring tools to identify and block suspicious interaction patterns that exploit context-aligned prefixes to poison generative recommendation APIs.
Authors
Zhuoxuan Zhang, Jinhao Duan, Xinyu Zhao, Jinduo Guo, Shuai Shao, Rui Li, Luke Simon, Zhun Deng, Tianlong Chen, Bo Sun
Abstract
Generative recommendation models based on autoregressive decoding have been adopted as alternatives to classical ranking-based systems. While their modeling flexibility has been widely studied, their resilience to data poisoning remains largely unexplored. In this work, we ask: how susceptible are generative recommenders to conditional targeted poisoning? Under a realistic setting where an adversary can inject limited malicious interactions and query a recommendation API as a black box, our empirical study shows that generative recommenders are vulnerable: injecting fewer than 1% synthetic interactions significantly increases the likelihood of promoting malicious targets under the trigger. To characterize this vulnerability, we analyze the synthesis of malicious interactions from two perspectives: (i) decoding alignment and (ii) prefix popularity. We observe that short, context-aligned prefixes that are both common under the model's induced transition dynamics and strongly linked to a frequent trigger consistently amplify targeted promotion. Motivated by this analysis, we develop AlignPoison, a black-box framework that discovers and samples context-aligned prefixes from API outputs. We conduct experiments on benchmarks in unimodal and multimodal generative recommendation settings. Results show that these prefixes outperform competitive baselines, improving ASR lift by up to 16.3 percentage points and highlighting a decoding-dependent, context-sensitive poisoning vulnerability in generative recommendation models.