Papers for

software engineers deploying ai models

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Decision model factory builds fast specialists with improved accuracy

Decision Model Factory: Building Specialized Sovereign Decision Models

Abstract: Decision needs vary with each application, language, and deployment environment. The Decision Model Factory turns those needs into specialized, sovereign models on demand. A brief recipe guides the training, calibration, verification, and evaluation of a selected backbone, delivering a deployable decision model under the operator's control. The same workflow creates a new specialist whenever the requirements change. Across Japanese decision tasks, compact deployment, and long-context passage ranking, five factory-built models improve upon their backbones; three surpass the strongest results in a fixed public Japanese benchmark comparison. A dedicated runner computes only permitted-answer probabilities and reuses shared context, more than doubling serving speed on matched multi-question workloads with essentially unchanged decision scores. Together, the factory and runner make fast, task-specific decision models practical to build and deploy whenever needed.

Mon 26 OctArtificial IntelligenceSoftware Engineering
The gist
Different tasks and languages need different decision models to work well. The authors created a system called the Decision Model Factory that quickly makes specialized models tailored to specific tasks and contexts. These models perform better than the original standard ones, especially on Japanese language tasks. They also made a runner that speeds up answering multiple questions without losing accuracy. Together, these tools make it easier and faster to build and use decision models that fit exactly what’s needed.
Open → 2610.34956v1

Upcycling pretrained large language models with efficient gated layer loops

UpLoop: Upcycling Large Language Models with Gated Loop

Abstract: Looped large language models trade parameter storage for recurrent computation, but most are trained from scratch. In practice, pretraining with looped depth is expensive, making post-training upcycling of a pretrained non-looped LLM an attractive alternative. We introduce \textbf{UpLoop}, a post-training recipe based on \emph{layer-wise} recurrence. Instead of looping a whole model or a middle block, UpLoop loops each selected layer locally, preserving the pretrained depth hierarchy and avoiding feature-space mismatch. To make loop execution useful, we add hidden-wise gates on the attention and MLP residual branches, so later loops act as progressive refinements rather than redundant rewrites. Empirically, UpLoop breaks the commonly observed saturation at two loops ($R=2$) in prior looped models. Starting from Qwen3-4B Base and applying SFT, UpLoop improves the average accuracy across 20 benchmarks by 6.17\% at $R=3$. We use $R=3$ as the main setting for the best accuracy--cost trade-off, while both native and extrapolated $R=4$ yield further gains.

Sun 18 OctMachine LearningArtificial Intelligence
The gist
Large language models (LLMs) are typically very big and costly to run. The paper introduces UpLoop, a new technique that makes pretrained LLMs work more efficiently by looping parts of their layers multiple times with special gates, instead of retraining them from scratch. This method keeps the model’s structure intact and improves performance by refining outputs on each loop. The authors show that using three loops with UpLoop can boost accuracy by over 6% on many tests, using a smaller computational cost compared to training a bigger model.
Open → 2610.25004v1