Papers for

llm platform developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Tools linked to large language models often misuse input data causing security risks

ToFu: Exploiting and Fuzzing the LLM-Orchestrator-Tool Trust Boundary

Abstract: Large language models (LLMs) are increasingly deployed as agents that interact with external software through tools, with an orchestration layer translating model outputs into structured tool invocations. This architecture creates an LLM-orchestrator-tool trust boundary where tool semantics may go unenforced and LLM-generated arguments may be incorrectly treated as trusted. We present ToFu, a tool-aware framework that automatically derives semantic security checks for LLM tools from their existing metadata and a CWE-guided sanitizer corpus. ToFu specializes these checks to individual tools and automatically generates fuzzing harnesses and semantic oracles using runtime introspection and outcome sanitization. We evaluated ToFu across 273 MCP servers containing 3405 tools and identified verified bugs corresponding to CWE weakness classes in 343 tools across 118 of the servers. Our findings show that real-world tools frequently fail to enforce their documented input semantics, making it unsafe to treat LLM-generated arguments as trusted.

Fri 9 OctCryptography and Security
The gist
Large language models (LLMs) sometimes work with external software tools by sending commands through an orchestration layer. The paper shows that these tools often do not properly check if the information from the LLM is safe and correct. The authors built a system called ToFu that can automatically test these tools and find security problems by looking at their documentation and trying many inputs. They found hundreds of real bugs, showing it’s risky to trust LLM arguments without proper checks.
Open → 2610.13651v1

Lossless tokenization stops forged control tokens in open language models

Nameless Tokenization: A Lossless Tokenizer-Level Defense Against Control-Token Forgery in Open-Weight LLMs

Abstract: Open-weight language models publish the strings their chat templates use to mark turns, roles and tool results, which the tokenizer maps back to the reserved identifiers the model obeys. Anyone who controls text in a prompt can therefore write a turn boundary indistinguishable from one the serving stack wrote. We audit 256 deployed chat tokenizers. All are forgeable, and the flag usually recommended as a fix leaves 56.6% forgeable because it misses the tool and reasoning markers agent systems rely on. We propose nameless tokenization, which leaves the control entries with a reserved identifier and no surface string, so the content encoder cannot emit one and message content reaches the model unaltered. Across five tokenizer families it reproduces the standard token stream exactly on attack-free data and lifts accuracy on a probe of delimiter-bearing text from 8.5% to 59.9%, where sanitizers lose it. Separating a delimiter's appearance from its identifier shows the identifier matters little against a bare task instruction, but carries most of a forged tool result and most of any forged turn once the system message tells the model to treat user content as data.

Tue 15 SeptComputation and Language
The gist
Language models use special tokens to mark parts of a conversation or tool outputs, and anyone can fake these tokens in prompts, tricking the model. The authors found that all 256 tested language model tokenizers can be fooled this way, even when common fixes are applied. They created nameless tokenization, which hides these special tokens from the input text so the model only sees genuine content. This method keeps normal behavior intact but greatly reduces errors caused by fake tokens in tests.
Open → 2609.16984v1