Papers for
ai model integrators
Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.
Mapping parameter-efficient fine-tuning methods in neural networks
A Categorical Atlas of Parameter-Efficient Fine-Tuning: Cones, Orbits, and Gauges
Abstract: Parameter-efficient fine-tuning has hundreds of methods, usually compared by parameter count and benchmark score. We compare them as maps. A pretrained checkpoint is a base point in weight space, and a weight-space method is a smooth map from a small parameter space that starts at that point. These maps form a slice category in which the frozen model is initial and full fine-tuning terminal. Three invariants of the map do most of the work. Its image near the base point governs expressivity, through the first-order image, the dimension, a min-cut bound on rank and whether that point is smooth or an apex. Its fibres, with the optimizer's metric, govern training dynamics. Base change moves the base point and covers splitting initialisations, quantisation, restarts and merging. We prove, for example, that zero-initialised LoRA, and Householder reflection adaptation at its default start under stated rank and parity conditions, begin at the apex of a cone; that no exact initialisation lets LoRA reach an update of rank above twice its own; that exactly orthogonal methods never change a weight's singular values in exact arithmetic, however often they are merged; that under Adam with no epsilon, weight decay or clipping and one shared schedule, LoRA+ from the standard zero initialisation follows exactly the trajectory of plain LoRA with its scale multiplied by the learning-rate ratio; that Adam with shared hyperparameters respects only the signed permutations inside LoRA's continuous symmetry group of full-rank factors; and that the activation function decides when a scaling adapter folds into preceding weights. We classify 382 methods from 138 papers by seven coordinates read off their formulas, with 403 simulation arrows and 485 obstructions. Every numbered result is proved in full, and results due to others are credited where we re-derive them.
Models vary widely in updating accuracy and error independence
Wavering Oracles: Selective Updating and Correlated Failures in LLMs and Their Implications for Scientific Workflows
Abstract: Scientific workflows increasingly use repeated queries, multiple models, and interacting agents. Reliability therefore depends on whether models preserve correct conclusions, accept valid corrections, and contribute errors that a selector can distinguish. Using SycoBench- 600 as a controlled measurement substrate, we evaluate these requirements through selective updating, defined by resistance to misleading suggestions and uptake of correct suggestions. The study covers ten models and 17,055 trajectories. Published models span 13.4 to 71.6 percentage points in selectivity. Under identical local evaluation, Qwen3-4B is selectively adaptive at 45.6 points, Gemma3-4B is destabilized at minus 14.1 points, and SmolLM3-3B follows both correct and wrong explicit suggestions, producing zero selectivity. Matched interventions identify model specific responses to doubt, authority, and explicit advice. Among seven published models, the best reaches 95.3 percent accuracy, plurality reaches 88.6 percent, and the oracle ceiling is 99.8 percent. Mean error correlation of 0.285 reduces seven models to an effective independent count of 2.58. A leave-one-stem-family-out reliability selector reaches 96.2 percent, recovering 67.7 percent of the plurality-to-oracle gap. These results establish selective updating, error diversity, and calibrated adjudication as jointly measurable design targets for multi-model scientific workflows.