Papers for

data pipeline engineers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

News copies trick large language models into trusting false claims

Copies Are Not Confirmations: How News Duplication Fools LLM Judges and Continual Learners

Abstract: Many news reports of a claim are copies of one source: wire stories, content shared within a media group, syndicated features and press releases. We ask whether LLM judges and continual learners treat such copies as independent confirmations. In 200,000 English CC-News articles, a MinHash detector links 18.9% to a near-copy from another outlet (precision 0.97 once boilerplate is removed); it links 92% of edited reprints to their originals but none of the LLM rewrites by phi4, Llama 3.2 or Gemini 2.5 Flash. Shown three identical reprints of a claim about an unknown person instead of one report, Llama 3.2 3B and phi4 14B raised the share of lies they accepted from 0.06 to 0.71 and from 0.21 to 0.90 and ignored a note that the reports came from one source; Gemini 2.5 Flash already believed a single report, and Gemini 2.5 Pro followed provenance notes and tables. In a simulated continual-learning stream, showing the judge one report per detected chain cut the copied lies it accepted by 0.10-0.24 across four judges, at a mean cost of no more than 4.2 percentage points of accepted truths. Against reworded copies, a test of how many of a report's named entities are absent from an earlier look-alike separates LLM rewrites from 100 labelled independent reports of the same event with AUC 0.98-0.99 and wrongly flags one; entity overlap alone does as well, TF-IDF and a current embedding model come close, and the test catches few natural same-source derivatives. With the miss patterns of generated rewrite networks, it cut the reworded-copy lies accepted from 0.67 to 0.20 (0.40 when the rewriter invents details). Counting reports or chains alone is not a reliable defence, the effect of provenance information depends on the judge, and colluding sources remain the open problem. Code and labels: https://github.com/Metalesaek/copies-experiment

Fri 6 NovMachine LearningComputation and Language
The gist
Many news articles repeat the same story copied from one source, which can trick AI systems into thinking multiple sources confirm the story. The authors found that large language models often accept false claims more when seeing repeated copies than a single report. Some models ignore notes about the duplication, while others use provenance info to avoid being fooled. They also tested ways to detect reworded copies by comparing named entities and found methods to reduce AI acceptance of false claims from copies. Counting sources alone is not enough to ensure reliability.
Open → 2611.09203v1

Model routing strategy cuts costs with truthful reports and context reuse

Truthful and Expected-Payment-Optimal Model Routing in Graph-Based Agentic Workflows

Abstract: Assigning routine tasks to cheaper large language model (LLM) providers can reduce generation cost, but switching providers may require processing accumulated context again. Keeping dependent tasks with the same provider can instead reuse eligible context. Graph-based workflows therefore couple routing decisions across tasks. Independent providers may also privately know their real-time unit costs and strategically misreport them. We formulate this routing problem as a single-parameter procurement auction and propose STAR (Strategy-proof Topology-aware Agent Routing), which minimizes virtual-cost-weighted graph workloads. Its allocation rule and payments, computed by integrating workload over possible cost reports, make truthful reporting utility-maximizing regardless of other providers' reports and cover each truthful provider's execution cost. With independent cost distributions and non-decreasing virtual costs, STAR additionally minimizes the user's expected total payment to providers. Controlled synthetic experiments separate the execution-cost benefit of topology awareness from the payment benefit of virtual-cost optimization: the former grows with fan-out and transferred payload, while the latter increases gradually along the tested prior-shape heterogeneity path. A paired sensitivity study quantifies how payment savings vary with prior misspecification while strategy-proofness is preserved.

Tue 13 OctComputer Science and Game TheoryComputation and LanguageMultiagent Systems
The gist
Assigning different parts of a multi-step task to various large language model providers can save money, but switching providers means redoing some work. The authors present a method called STAR that decides how to route tasks to providers by considering the workflow structure and the providers’ costs, encouraging them to honestly reveal their prices. This approach not only saves costs by reusing previous work when possible but also ensures the total payment to providers is minimized on average. Experiments show the benefits come from both smarter task grouping and optimizing payments under provider cost uncertainty.
Open → 2610.19316v1

Workflow graph structure improves prediction of task resource use

About the Influence of Workflow Topology on Task Intensity Prediction through Graph Learning

Abstract: Efficient resource provisioning for large-scale workflows on cloud infrastructures is a critical performance engineering challenge. These workflows are often structured as directed acyclic graphs (DAGs), where under-provisioning can cause critical bottlenecks and over-provisioning leads to unnecessary costs. Accurate, task-level prediction of resource intensity (e.g., CPU load and memory usage) is essential for mitigating these issues. While task-level features are commonly used for prediction, the performance impact of the workflow's overall topological structure is often overlooked or assumed. The central question of our work is: To what extent does what part of the DAG topology influence task-level resource intensity, and what is the most effective way to model this influence? This paper presents a comprehensive benchmark to systematically quantify the impact of graph topology on task intensity prediction. We evaluate and compare a spectrum of modeling approaches. Our findings demonstrate that topology is a critical feature for accurate prediction. Models incorporating important topological information, even through simple handcrafted features, significantly outperform baseline models. We show that graph-native models provide the highest accuracy, achieving low mean absolute errors for both CPU and memory predictions, and can still be combined with simple topological features that they do not learn for better performance.

Wed 30 SeptDistributed, Parallel, and Cluster ComputingMachine Learning
The gist
Workflows in cloud computing are like maps showing the order of tasks. Predicting how much CPU or memory each task will use helps avoid slowdowns or wasted resources. This paper by the authors shows that not just task details but also the overall shape of this map (the workflow's graph) matters a lot. They tested various methods and found that using the graph's structure helps make much better predictions, especially when combined with models designed for graphs.
Open → 2609.39481v1

Different evidence types affect AI prompt improvement results

What Should the Reflector See? An Empirical Study of Evidence in Reflective Prompt Optimization

Abstract: Reflective prompt optimization revises instructions using examples of a model's behavior, but which evidence the reflector should receive remains unclear. We study evidence composition, visibility of examples, candidate selection and domain-knowledge policy within a single-parent Pareto-guided search. Using Qwen3.5-9B as both task model and reflector, we evaluate nine reflection strategies on five datasets. From these experiments, we find three distinct patterns. For performance improvement, Failures-only produces the largest mean test gain (+8.0 percentage points), while Balanced-mix and No-examples+Val share the best mean performance rank. For reflection effectiveness, No-examples achieves the best rank for improving sampled parents, yet yields only a 1.4-point mean test gain: local reflection success does not necessarily produce a stronger final prompt. For overfitting assessment, Failures-only and Balanced-mix share the lowest mean calibration-gap rank, while larger gaps on GPQA and IFBench show that calibration gains can overstate held-out improvement. This gap is a descriptive indicator, not a direct measure of overfitting. Together, these results show why reflection strategies should be assessed separately on final performance, parent improvement and calibration-to-test transfer.

Sat 26 SeptComputation and LanguageMachine Learning
The gist
When improving instructions given to AI models, it’s unclear which examples the model should see to do better. The authors tested different ways of showing past results to the AI and found some strategies help more than others. For instance, showing only the failures improves test scores a lot, but sometimes success nearby doesn’t mean better final results. They recommend evaluating improvement methods by looking separately at final scores, local improvements, and overfitting signs.
Open → 2609.32452v1