Papers for

machine learning platform operators

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Input optimization creates steering prompts that guide language model behavior

Creating Steering Vectors via Input Optimisation

Abstract: Activation steering influences a language model's behaviour by adding a steering vector to its residual stream during generation, but it requires white-box access to modify the hidden states at every generation step. We introduce ActGCG, a modification of Greedy Coordinate Gradient which optimises a suffix to match the target activations obtained by adding a steering vector to the residual stream. We further introduce universal suffixes trained across many prompts at once, a model-translation step that rewrites the suffix into human-readable text midway through optimisation, and a soft-suffix variant to remove the discrete token space constraint. We study two behaviours: steering a model to answer in German, and anti-refusal steering for jailbreaking. For German, a discrete suffix reproduces the steering vector's behaviour almost perfectly and transfers well across prompts and models. For anti-refusal, the equivalence is weaker, succeeding under a phrase-avoidance metric but only partially under the stricter HarmBench classifier. Our results show that a steering vector's behaviour is reachable from the input, with no access to model internals at generation time, up to a loss that depends on the concept being steered.

Thu 12 NovComputation and LanguageMachine Learning
The gist
Language models can be guided by adding special signals inside their calculations, but this usually needs deep access inside the model. The authors developed a way to create input text sequences that cause the model to behave like it was steered, without needing to peek inside during use. They tested this by making the model reply in German or avoid refusing certain prompts. Their method works well for the German case and somewhat for the anti-refusal case, showing that input text alone can mimic steering signals with some limits.
Open → 2611.17481v1

ModelShield catches hidden malicious actions in shared ml models

I Don't Care How You Hide It, I Care What You Do: Runtime Detection of Malicious ML Models

Abstract: Pre-trained machine learning (ML) models shared through public platforms can be weaponized to execute malicious payloads during loading or inference. Existing scanners prove inadequate for detecting malicious models as they rely on static analysis and function denylists, support limited serialization formats, and miss attacks that replace denylisted functions with equivalent alternatives, exploit unsupported loading paths, or abuse legitimate framework APIs. To address this problem, we present ModelShield, the first runtime behavioral detection system for malicious ML models that attributes observed system-level behavior to specific model operations. It combines application-level instrumentation of ML framework APIs with eBPF-based syscall tracing, tracking model behavior across spawned threads and processes to separate model-triggered activity from environment noise. Attributed syscall sequences are classified using 195 detection patterns mapped to MITRE ATT&CK techniques. We implemented ModelShield to support 17 ML frameworks and 36 formats. On an independent dataset, it is the only evaluated tool to achieve zero false positives, with false negatives limited to payloads that produce no malicious runtime behavior. It detected all 29 published evasion variants spanning function substitution, loading-path obfuscation, and TensorFlow API abuse, where no existing scanner achieved complete coverage. Across 145,000 models scanned from public platforms, it captured 41 malicious models on Hugging Face, including a reverse-shell payload and a payload that performed host reconnaissance and outbound exfiltration. Among these, ModelScan, PickleScan, Fickling, and ModelAudit detected only 29%, 56%, 80%, and 93%, respectively. Despite runtime overhead, ModelShield has a median analysis time of 12.3 seconds, with 90% of scans completing within 35.5 seconds, making it practical for hub-scale screening.

Thu 5 NovCryptography and Security
The gist
People can share machine learning models online that secretly do harmful things when used. The authors show that current detection tools miss many of these sneaky models because they look only at fixed code pieces and miss tricks like replacing banned functions with similar ones. They created ModelShield, a system that watches what the model actually does while running, tracking specific behaviors linked to the model’s actions to find malicious activity. Their tool caught all known tricky attacks and found harmful models on popular platforms without false alarms, and it runs fast enough to screen large collections of models.
Open → 2611.08745v1

Generative recommendation improves search by reallocating beam budget early

Where Beam Search Loses Top-Ranked Items:First-Step Coverage in Generative Recommendation

Abstract: Generative recommendation can reproduce a wider beam's ranked output with roughly half the prefix-capacity budget by reallocating retention capacity across semantic-ID positions. We identify a \emph{first-step coverage bottleneck}: across six full-test settings of the TIGER generative recommender spanning three domains and two identifier constructions, every pruned exact top-10 path is first lost at the initial decoding step. An exact-catalog reference isolates search losses and guides a controlled budget intervention. Moving width to the first step raises Search Recall@10 by 0.96--10.90 percentage points at Beam-50's unchanged budget. Across all 77,373 test users with residual-quantized variational autoencoder (RQ-VAE) tokenization, the reallocated schedule reproduces Beam-100's ordered top-10 at nominal budgets of 151 rather than 301, without retraining or changing identifiers or scores. Matched-output timing saves 6.35--6.39\% of decoder latency; profiling identifies the execution costs that limit the translation from reduced search work to runtime savings. Larger-candidate experiments reveal how output size changes retention requirements. These findings make budget placement an actionable serving decision: locate ranking loss, redirect search capacity, and preserve downstream inputs before increasing uniform beam width.

Sat 31 OctInformation Retrieval
The gist
When computers suggest items, they look through possible options step-by-step using a method called beam search. This paper finds that many top item suggestions get lost right at the first step of that search process. By shifting more search effort to this initial step, the authors show it's possible to keep better recommendations without using more resources or retraining models. This adjustment speeds up the recommendation process while keeping quality high.
Open → 2611.01072v1

Looped transformers improve performance without extra training cost

UpLoop: Training Looped Transformers via Upcycling at No Extra Pretraining Compute

Abstract: Looped language models can strengthen reasoning through repeated computation with shared weights, increasing execution depth without adding independent parameters. However, each additional pass increases training floating-point operations (FLOPs), making these gains more expensive to obtain. To address this cost, we propose UpLoop, a pretraining schedule that uses upcycling to train looped Transformers while exactly matching a standard dense baseline in final parameter count, total training tokens, and training FLOPs. Specifically, UpLoop begins with a shallow Transformer, progressively grows its depth by copying learned layers, and uses the computation saved early to support repeated execution of a shared middle block later in training. Experiments across model scales show improvements in language modeling and downstream performance, with the average score across 22 downstream tasks increasing from 22.99 to 27.52 at 1.3B parameters. These gains also extend to mathematical reasoning after supervised fine-tuning.

Fri 30 OctMachine LearningComputation and Language
The gist
Training language models to think more deeply by repeating certain parts usually means more computing power is needed. The authors propose UpLoop, which starts with a small model and grows it by reusing layers, balancing training so it doesn’t use more computing resources than usual. This approach helps the models get better at understanding language and solving problems, like math, without extra cost in training. Their tests showed that this method improves results across many language tasks and mathematical reasoning after fine-tuning.
Open → 2610.41720v1

Momentum-free method improves accuracy and cuts memory for large language models

MUON-RT Reusing the Teacher's Trajectory for Momentum-Free Muon for LLM Self-Distillation

Abstract: Self-distillation methods for large language models with an exponential moving average (EMA) teacher already retain a history of student updates. Muon nevertheless stores a separate momentum buffer, increasing persistent training memory. We propose Muon with Reused Teacher (Muon-RT), which reuses the teacher-student displacement to eliminate momentum buffers for hidden weight matrices. We show that this displacement encodes an exponentially weighted history of applied updates and smooths temporally uncorrelated noise under a local model. Muon-RT removes its radial component, normalizes it spectrally, and mixes it with the current gradient after Newton-Schulz (NS) orthogonalization. Under explicit regularity and alignment assumptions, we establish linear contraction of an expected objective-gap bound toward a residual error level. With Self-Distillation Fine-Tuning (SDFT) on Qwen2.5-7B-Instruct, Muon-RT improves average accuracy over AdamW by 2.6 points with 42% less persistent memory, and over momentum Muon by 5.4 points with 27% less persistent memory. With Self-Distillation Policy Optimization (SDPO) on Olmo-3-7B-Instruct, it improves average accuracy over AdamW by 2.0 points.

Fri 30 OctMachine LearningArtificial IntelligenceComputation and Language
The gist
Large language models learn better by teaching themselves using past knowledge, but this often requires extra memory to store data about past changes. The authors propose a new technique called Muon-RT that reuses information from the teacher model to remove the need for extra memory buffers while still keeping past learning updates in mind. This approach improves the model’s accuracy while using less memory during training. They tested it on two language models and found it outperformed previous methods with less persistent memory use.
Open → 2610.41104v1

DolphinRouting improves disaggregated large language model serving efficiency

DolphinRouting: Cluster-Level Binary Routing for Disaggregated LLM Inference Serving

Abstract: Prefill-decode (PD) disaggregation isolates the two phases of large language model serving, but request-level path selection must account for prefill work, cache reuse, and decode demand. We present DolphinRouting, which learns this choice from end-to-end SLO feedback under a given P/D resource allocation. Requests are clustered by raw prompt length, cache-adjusted input lengths on both worker pools, and output-length information. Offline policy search learns one binary action per cluster. Online feature extraction, cluster assignment, and table lookup select local or remote prefill. We evaluate DolphinRouting on a real PD-disaggregated cluster using three public benchmarks, a mixed workload, and real data. Across the mixed workload and real data, DolphinRouting with $k$-means reduces P90 TTFT by 5.7--7.9\% and P90 TPOT by 2.6--10.1\% relative to Dynamo.

Thu 29 OctMachine Learning
The gist
Serving large language models often involves splitting work into two parts: preparing the input (prefill) and generating the output (decode). Choosing where to handle these parts—locally or remotely—is tricky because it depends on many factors like cache use and request size. The authors designed DolphinRouting, a system that learns the best place to do prefill work for different groups of requests, improving response speed and efficiency. They tested it on a real cluster and showed it reduces latency compared to previous methods.
Open → 2610.40292v1

Self improving agents lose safety safeguards when rules overlap

Masked Erosion: How Acceptance Gates Lose Redundant Safeguards in Self-Improving Agents

Abstract: Self-improving LLM agents edit their own harness, the prompts and rules around a fixed model, and keep an edit only if an acceptance gate scores it higher on a batch of tasks. Such a gate sees what the harness does under the current model, not what it would still prevent if one safeguard failed. When safeguards overlap, deleting one is masked: the others still block the harm, so the gate decides on task success alone and redundancy erodes unseen. We measure harm for every subset of four written safeguards in two agent environments and run 411 improvement loops under ten gate rules, mostly as controlled deletion experiments, plus three runs of the GEPA optimizer. A deletion behind another working safeguard adds almost no observable harm. A paired significance test blocks deletions that pass by chance but not deletions that genuinely raise task success, and a larger evaluation batch lets it delete cheaper safeguards: at the same cost, 20-task batches kept every harness protected while 40-task batches lost all six. Large harm audits prevent exposure but not the loss of depth, which turns into harm when the harness is reused with a weaker model. GEPA kept every safety rule verbatim, yet two of its three final harnesses misused credentials under a weaker model. A statistical model frozen before the runs predicts where each gate's protection breaks as safeguard cost and batch size change, and freezing safety text or auditing one-layer ablations preserves depth.

Wed 28 OctArtificial Intelligence
The gist
When large language model agents improve themselves, they test changes using a gate that looks only at the overall success of tasks, not at hidden safety safeguards behind the scenes. The authors found that if several safeguards cover the same risk, deleting one can seem harmless because the others still work, which hides the problem. Over many tests, they showed how this ‘masking’ causes agents to lose important overlapping safety checks, making them vulnerable later if a weaker model is used. The paper also shows that traditional audits don’t catch this loss, but freezing safety details or testing one safeguard at a time can help.
Open → 2610.38392v1

Agent-conditioned routing doubles diverse solution paths in AI models

Agent-Conditioned Routing: Diverse Reasoning Paths Within One Frozen Model

Abstract: Generating several solutions can expose alternative methods, but additional trajectories often paraphrase one dominant approach. We investigate whether the conditional capacity of a frozen sparse mixture-of-experts (MoE) model can provide such diversity from one problem prompt, without user-assigned roles, prescribed methods, fine-tuning, or model copies. Agent-Conditioned Routing (ACR) assigns $m$ parallel trajectories persistent, untrained shared/private expert-eligibility masks while preserving the learned, input-dependent token router. On a screened 126-question multi-mode mathematics and physics benchmark, with model judges reading the reported derivations in full, we count the number of problems yielding at least two distinct primary methods among soft-correct solutions in four trajectories. ACR reaches approximately twice as many such problems as the strongest temperature setting under GLM 5.3 (37.3 versus 18.7), and 1.67 times as many under GPT-5.6 Sol (45.00 versus 27.00), at matched output budgets. A blinded four-expert audit on a stratified sample of 30 problems also finds more multi-method solutions with ACR on both backbones, with lower correctness. Matched-eligibility ablations show modest multi-method count advantages for ACR, while token-resampled masks attain higher correctness. An informed sequential prompt elicits more diversity still, using previous solutions and explicit requests for alternative methods. ACR adds about 160 KiB of mask state. In a three-problem microbenchmark capped at 1,024 tokens, its latency and peak VRAM usage remain within 3% of batched temperature sampling.

Tue 27 OctArtificial IntelligenceMachine LearningMultiagent Systems
The gist
Generating multiple ways to solve math and physics problems often results in similar answers. The authors studied a frozen AI model that can create diverse solutions without changing the model or using special prompts. They introduced Agent-Conditioned Routing (ACR), which assigns different fixed masks to create varied reasoning paths in parallel. Tests show ACR produces almost twice as many distinct solution methods compared to traditional techniques, with minimal extra computation.
Open → 2610.37375v1

Stronger judges accept more manipulated outputs in AI evaluation

Stronger Judges Do Not See More: Deployment-Integrity Blindness in LLM Evaluation

Abstract: LLM-agent evaluation reports behavioral scores but leaves the identity of the deployed runtime stack unreported. A tampered deployment that preserves evaluated outputs passes every output-only evaluator. We introduce DIB-Bench, a benchmark that measures this deployment-integrity blindness across 7,560 scored examples, six tamper types, and an evaluator ladder from regex scorers through LLM judges to GPT-5-nano. Stronger evaluators accept more tampered outputs: mean acceptance rises from 88.6% (rule-based) to 94.1% (GPT-5-nano), and a trigger-conditional tamper invisible on the evaluation suite produces 48-100% harmful outputs, rising with model capability. A PAC-style bound shows that tampers with zero output shift are undetectable from outputs at any sample size; a manifest-hash identity channel detects them with zero samples. Behavioral fingerprinting identifies the model component above chance (76.7%) but covers one of six stack components. We also propose a graded deployment-identity descriptor for evaluation reports.

Thu 15 OctCryptography and SecurityArtificial Intelligence
The gist
Evaluating large language models (LLMs) often means checking only their final answers, not how they run behind the scenes. The authors found that even very advanced evaluators can’t spot when the system is secretly tampered with, as long as the outputs look normal. They created a test benchmark to measure how often these tampered outputs slip through undetected, and interestingly, better evaluators were more likely to accept them. They also showed that some hidden changes are impossible to catch by just checking answers, but verifying the system’s identity directly can detect them right away.
Open → 2610.21135v1

Measuring debate shifts in AI model answers on multiple-choice tests

Measuring Collapse and Correction in Homogeneous-Panel LLM Debate

Abstract: Multi-agent large language model (LLM) debate is often evaluated by whether final answers improve, but movement is not necessarily improvement: the same discussion can rescue an initially wrong majority or destroy an initially correct one. Standard final-accuracy evaluations conflate these opposing mechanisms. We introduce an auditable protocol for homogeneous debate on multiple-choice questions (MCQs) that records each run as a transition ledger over collapse, correction, onset, and signed intervention utility. On 6,925 MMLU-Pro debates, the protocol identifies 253 collapses and a parallel correction ledger that changes how interventions should be judged. Replay experiments reveal the central tradeoff: a leave-one-model-out probe-gated freeze prevents 29 collapses but loses 108 corrections under equal weights, so collapse prevention alone can recommend the wrong policy. A compact pre-debate 8-probe screen is a triage signal: its unadjusted family-level association with conditional-collapse risk is high (G=7, Spearman rho=0.893, exact two-sided p=0.0123), but initial-majority accuracy is a close comparator (rho=0.821; family partial rho=0.767, p=0.0877), so we do not treat it as calibrated or capability-adjusted prediction. Round-level traces localize many collapses to the first debate round, where early disagreement can precede both harmful cascades and useful recovery. We release replayable schemas, coders, audits, cost cards, and zero-API rebuild scripts so future model-scaffold rows can be compared under the same denominators and signed utility ledger.

Mon 28 SeptComputation and Language
The gist
This paper looks at how AI models argue over multiple-choice questions and whether their discussions really help or hurt the final answer quality. The researchers find that just seeing the final result improves doesn't tell the full story, because discussions can either fix mistakes or cause new errors. They develop a detailed way to track these changes during debates, showing when answers collapse or get corrected. Their tools help figure out when debate helps and when it might backfire.
Open → 2609.35279v1

Neural network training verified cheaply without trusting trainers

Training Witnesses: Trusting the Training without Trusting the Trainer

Abstract: Progress in machine learning cannot outpace our ability to verify it. With an explosion in papers today, every scientific claim rests initially on trust in the trainer, leading to uneven evaluation, baselines, and forestalling of reliable progress. Traditionally, the burden of verification falls on the reader, who must reproduce expensive training runs. This strategy is impractical due to an explosion in slop contributions, diversity of methods, and the sheer compute required. We put the burden of proof where it belongs, on the trainer, and in the process also cut the overall cost of verification significantly. We introduce Witnesses, a method for certifying training, data usage and evaluation in a neural network training run. Our key insight is that fast behavioral fingerprints with occasional replay challenges are sufficient for auditing neural network training. Our method is applicable at scale with minimal overhead to the trainer, is cheap for the verifier, rejects bad training runs with amplifiable probability, and allows for exact queries of both data inclusion and exclusion. We test our method on language model training runs from 100M to 2B scales, across DDP and FSDP, and demonstrate this minimal overhead. We also introduce a self-regulating leaderboard of "auto-certified" training runs that enables shared baselines and progress. We invite the community to participate in the leaderboard to improve reproducibility in machine learning.

Sun 27 SeptMachine LearningCryptography and Security
The gist
Machine learning results depend on trusting the person who trained the model, which can lead to mistakes or unfair comparisons. The authors propose a way to check if training was done correctly without redoing the entire process. Their method uses quick behavioral checks and occasional tests to confirm the model's training and data use. This makes it easier and cheaper to verify models and helps create trusted benchmarks for progress.
Open → 2609.33915v1