AI summaryⓘ
The authors study how large language models can turn their problem-solving experiences in scientific computing into lasting skills that help with future problems. They introduce SciConsolidate, a method that learns from both successes and failures without needing answer keys, and uses a stronger model to turn abstract knowledge into concrete code for training weaker models. Their experiments show that without this step, smaller models struggle to use the learned procedures effectively, revealing a gap between understanding and execution. After converting the knowledge into usable code, smaller models improve significantly, demonstrating a way to build models that get better with experience in scientific tasks.
large language modelsscientific computingexperience consolidationprocedural knowledgeself-finetuningabstraction-execution gapruntime feedbackcode supervisionmodel improvementQwen models
Authors
Liwei Dong, Jiahao Zhao, Nan Xu
Abstract
Large language models increasingly solve scientific-computing tasks, but executable feedback from one problem rarely becomes durable capability on subsequent problems. We study scientific-computing experience consolidation: converting verified runtime experience into transferable procedural knowledge and persistent model improvement. This setting presents two challenges: trajectory-derived artifacts may encode source-specific repairs rather than cross-task computational mechanisms; and a weaker target model may be unable to operationalize an otherwise valid abstract procedure - an abstraction-execution gap. We introduce SciConsolidate, which contrasts verified successes and failures to induce cross-task procedures, selects them through a development-validation gate, and uses failure-informed, answer-free query synthesis to expand the consolidation data without requiring pre-existing reference answers. Because the target model may not directly execute these abstractions, a stronger model concretizes them into executable code supervision for standard, procedure-free SFT; a matched no-procedure teacher branch isolates the value of procedural guidance. On SciCode, runtime procedure injection improves Qwen3.6-27B by +3.85/+6.26 sub-step/main-problem points, but yields almost no aggregate main-problem gain for Qwen3.5-9B, providing operational evidence of the abstraction-execution gap. After procedure-guided concretization, the 9B student improves under procedure-free deployment by +3.89/+6.25 points over the no-procedure SFT control and by +5.62/+11.25 over the original 9B model. These results establish an experience-to-capability pathway for scientific computing and provide a practical starting point for scaling self-improving scientific assistance.