Papers for

software developers in ai tools

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Knowledge evolution improves protein design with modular strategies

KnowEvo: Knowledge Evolution for Protein Optimization

Abstract: Large language models (LLMs) have shown promise in scientific discovery, yet systematically accumulating and refining reusable scientific knowledge remains challenging. Existing approaches primarily improve LLM-based problem solving through parameter optimization or feedback-driven refinement of prompts, programs, and solution trajectories, leaving acquired expertise either embedded in model weights or scattered across iterative problem-solving processes. As a result,knowledge is difficult to systematically inspect, evaluate, and refine. We propose KnowEvo, a framework that treats externalized knowledge itself as the optimization target. Our key insight is that scientific knowledge is inherently modular: multiple interacting mechanisms jointly determine the design outcomes. This makes knowledge evolution fundamentally challenging, as improvements cannot be attributed to individual components in isolation. To address this, KnowEvo represents knowledge as executable expertise blocks and organizes their refinements into strategy mutation trees, enabling structured exploration over a combinatorial knowledge space. The framework combines a Thompson-style forward allocation strategy that adaptively distributes evaluation budgets across strategy variants with a backward revision process, in which a code agent operates in an isolated sandbox to translate rollout evidence into localized program updates. This design enables persistent accumulation, evaluation, and refinement of mechanistic expertise across iterative experimentation. We evaluate KnowEvo on protein optimization tasks including stability and solubility. Results show that knowledge evolution consistently improves over initial strategy blocks and outperforms monolithic program evolution and prior LLM-guided evolution baselines across metrics such as pass@k, MRR, MAP,and NDCG@5. Code is available at https://github.com/recynie/KnowEvo.

Fri 6 NovMachine Learning
The gist
Designing better proteins can be tricky because many factors work together in complex ways. The authors propose KnowEvo, a framework that treats scientific knowledge as building blocks that can be improved over time. These blocks can be tested, refined, and combined in different ways to get better protein designs. They show that this approach works better than previous methods on protein tasks like stability and solubility. This makes it easier to improve and understand how protein design knowledge evolves.
Open → 2611.09013v1

Two-layer evaluation improves assessment of text-to-SVG faithfulness

SVGFaith: A Two-Layer Meta-Evaluation of Faithfulness Metrics for Text-to-SVG Generation

Abstract: Text-to-SVG systems are commonly evaluated by applying raster-image metrics to rasterised outputs, yet whether these scores capture fine-grained prompt faithfulness remains unclear. We introduce SVGFaith, a two-layer meta-evaluation framework in which the more faithful candidate of each pair is determined by construction or by human judgment. SVGFaith-Diagnostic contains 1,842 controlled contrastive pairs across eight capability axes, split evenly between two directions: two renderings compared against one prompt, and two descriptions compared against one rendering. SVGFaith-Arena contains 1,327 human-annotated comparisons of real generator outputs. On SVGFaith-Diagnostic, CLIP and three other image--text embedding metrics reach at least 0.789 pairwise accuracy on object identity but at most 0.672 on spatial relations and attribute binding, where chance is 0.5. We further introduce SVGFaith-Score, an interpretable pointwise metric. A domain-tuned decomposer turns each prompt into a dependency graph of atomic visual requirements, a visual question answering model scores whether each requirement is satisfied, and a requirement's score is discounted when its prerequisites score low. In addition to the overall score, the per-requirement scores show which requirements are judged unsatisfied and which inherit discounts from their prerequisites. SVGFaith-Score significantly outperforms all nine baselines, including a Qwen3.5-122B-A10B judge, on Arena and in both Diagnostic directions. Code and data: https://github.com/Yukinonooo/SVGFaith.

Wed 4 NovComputer Vision and Pattern RecognitionComputation and Language
The gist
It is hard to check if pictures made from text match the text well, especially when pictures are complex SVG images. The authors made SVGFaith, a new way to compare and score how well captions and images match by using carefully made test pairs and human judgments. They created a diagnostic dataset that tests specific abilities like recognizing objects and spatial relations, showing existing scores work well on objects but struggle with details. They also built SVGFaith-Score, which breaks down prompts into small parts and judges each part, giving a clearer and more accurate faithfulness score.
Open → 2611.06048v1

Mathematical guarantee for training mixtures of linear expert models

Provably Learning Mixtures of Linear Experts

Abstract: Mixture-of-Experts (MoE) models are widely deployed in modern machine learning. However, few convergence guarantees exist for gradient-based training of MoEs: the objective is nonconvex, and even simple mixtures trap first-order methods in spurious local optima. We analyze a canonical gated mixture featuring linear experts and a gate induced by Gaussian clusters in the input space, and give, to our knowledge, the first global convergence guarantee for jointly training the gate and the experts of such a model by gradient EM on its likelihood. Given a logarithmic overparameterization relative to the true number of experts $m$, gradient EM converges from randomly initialized gates and least-squares-initialized experts to any prescribed accuracy in polynomially many iterations, under a separation condition on the gate centers. Our analysis establishes a gradient-dominance inequality for the coupled dynamics by decoupling the loss into per-cluster subproblems. Empirical results validate the predicted $m\log m$ phase transition and suggest that the least-squares warm start required by our analysis is not needed in practice: randomly initialized experts perform nearly as well. We also find that leaving the mixing weights unconstrained concentrates routing onto the true number of experts, whereas load-balancing penalties spread it, so that the fitted model can be pruned to a fraction of its width without loss of accuracy.

Tue 3 NovMachine Learning
The gist
Mixture-of-Experts (MoE) models break complex tasks into parts handled by different expert models, but training them is tricky and can get stuck. The authors analyze a basic form of MoE where the input groups follow Gaussian patterns, proving for the first time that a common training method can find the best solution from random starts under certain conditions. Their math breaks the problem into simpler parts for easier analysis, and experiments show their approach matches predictions and works well even without some technical starting steps. They also find ways to simplify the model by focusing on the truly used experts without losing accuracy.
Open → 2611.05184v1

Mode dynamics reveal how neural network learning changes outputs

Mode Dynamics: A Framework for Neural Network Learning

Abstract: We introduce Mode Dynamics, a framework that connects coordinated parameter motion during neural network learning to changes in model outputs. Starting from a recorded parameter trajectory, we decompose its updates into collective modes and map each mode to its state-dependent output effect. The framework organizes this link through three mechanisms. Decomposition identifies collective modes in parameter motion and their time-varying amplitudes. Reconstruction tests whether the effects induced by those modes recover local and accumulated output change. Concentration quantifies how behavioral contribution is distributed across modes. Controlled scalar regression and next-token experiments directly test these mechanisms. They show that small sets of leading modes reconstruct behavior, induced effects depend on the training path, and modal complexity varies with task and data. We then evaluate the framework on held-out image and language-model outputs using CIFAR-100 autoencoding and a public Pythia-70M pretraining trajectory. Across these settings, the results connect compact modal subspaces of recorded parameter motion to evolving behavior.

Thu 29 OctMachine Learning
The gist
Neural networks learn by adjusting many parameters, but understanding how these changes affect what the model outputs is complex. The authors propose Mode Dynamics, which breaks down parameter updates into collective patterns (modes) and links each to how it changes the model's behavior. They show that a few key modes explain most output changes and that these modes depend on the task and data. This framework helps clarify how learning progresses inside neural networks.
Open → 2610.40143v1

Hidden model reasoning signals detected but not used for answers

Present but Hard to Read: Probing Latent Reasoning with Minimal Pairs

Abstract: Probing studies of latent reasoning often read a model's state where the answer begins and take a failed probe as evidence that a value is absent. We test this with a digit copied from the question, in the looped model Ouro-1.4B and the standard transformer Qwen2.5-1.5B. Comparing prompts that differ only in that digit shows it is present at the last prompt token, in a code shared across questions (readout about 0.9). A probe that sees one prompt at a time, the usual setting, barely reads it: 0.36 in Ouro and 0.19 in Qwen even after 5,249 training questions. The digit's change is small rather than crowded: random noise lands on equally busy directions, while a random word in its place moves the state far more and is readable. Since every varied token leaves a trace, a successful probe is ambiguous too. We give a protocol, including a synthetic null for generic mixing, for testing computed values against copies and mixtures of the inputs. In pre-registered studies of two-step problems it finds a small signal for the unwritten intermediate a+b just before Ouro writes the answer's last digit, and its sign follows the operation, though mixing alone produces much of that sign-following part. Holding that value at another problem's from the first loop on does not move the written digit toward the other problem's, while the same patch of the answer does: the value is readable but not used.

Tue 20 OctMachine LearningComputation and Language
The gist
Machine learning models often seem to hide the steps they take when solving problems. This study shows that while tiny clues about intermediate calculations exist inside the model, these clues are hard to read and don’t actually influence the final answer. The researchers tested this by inserting minimal changes and checking if the model’s internal state reflected these changes clearly. They found the signals are present but very faint and sometimes mixed with unrelated information, meaning current methods might overestimate what the model 'knows' internally.
Open → 2610.28498v1

Transformer model replicates causal discovery update steps accurately

Executing Causal Structure Learning with Linear-Attention Transformers

Abstract: Transformers can execute algorithms on data given in their input. We ask whether they can do the same for causal discovery. We study a standard continuous method that repeatedly updates a candidate causal graph while enforcing acyclicity. We explicitly construct a fixed-weight transformer whose forward pass exactly reproduces one update of this method, so repeated blocks reproduce its optimization trajectory. The transformer carries the current graph and the algorithm's multiplier between updates. We show that retaining the multiplier is essential for exact execution, since different multiplier values can lead to different next updates. We also give conditions under which, within a fixed stage, the number of updates needed to reach a target accuracy can be computed in advance and rounding errors stay bounded as depth grows. Experiments show that the constructed block agrees with a reference update to floating-point precision, while arithmetic replay on synthetic data and seven published benchmark network topologies inherits the reference solver's successes and failures. This separates accurate algorithm execution from accurate causal recovery. In contrast, the ordinary attention models tested under our training budgets do not reliably execute the update or transfer to larger graphs. Whether gradient training can learn an executor in the architecture class of the construction remains open.

Wed 7 OctMachine Learning
The gist
Finding relationships that cause effects is hard, especially when computers need to figure it out step by step. The authors show that a special kind of AI model called a transformer can perfectly mimic one step of a known method for discovering cause-and-effect links in data. This approach repeats the steps by passing information forward multiple times, holding extra details needed for accuracy. They find normal transformer training doesn’t learn this behavior well, leaving open whether such models can be trained to do this on their own.
Open → 2610.10395v1

Vision language models improve self-training with program-verified checking

Program-Verified Self-Evolution for Vision-Language Models

Abstract: Self-evolving vision-language models train on questions they generate from unlabeled images. Since these questions have no gold answers, prior methods label them by majority vote over sampled answers or by a model judge. In a human evaluation, we find that 24\% of majority-vote labels and 18\% of model-judge labels produced during self-evolution are wrong. To address this problem, we present Verifiable QA Generation for Self-Evolving Models (VQS), which changes how the model judges answers. Instead of voting on an answer, the model parses each image into a structured record, such as a scene graph, a chart table, or a diagram graph. Fixed programs then write a question from the record and compute its answer. The model still acts as a visual checker, but it only confirms the individual facts the program reads, one short claim at a time. These claim-level checks select the parser's training targets, so the parser also improves without labels. Human raters find 94\% of VQS answers correct, against 76\% for majority voting. Across ten benchmarks, VQS improves Qwen3-VL by up to 3.18 points at the 2B, 4B, and 8B scales and outperforms the strongest self-evolving baseline at each. Gains keep growing over three training rounds, reaching 3.84 points at 2B. Code is released at https://github.com/ahmedheakl/VQS

Sun 27 SeptComputer Vision and Pattern RecognitionArtificial IntelligenceComputation and Language
The gist
Vision-language models learn by asking questions about pictures without needing human-provided answers. The problem is that guessing the correct answers can be wrong a lot of the time. The authors propose a new way where the model breaks down images into structured data like graphs and uses fixed programs to create questions and verify answers more accurately. This method leads to much more reliable answers and better overall model performance.
Open → 2609.33855v1