Beyond Tier Labels: Role- and Deployment-Dependent Model Substitution in Multi-Call LLM Workflows
2026-08-10 • Multiagent Systems
Multiagent Systems
AI summaryⓘ
The authors study how to best use different levels of large language models (LLMs) when many calls depend on each other in a workflow. They find that deciding where to use a stronger model depends not just on the model tier but on the specific place and context within the workflow. By separating the choice of which model to use from the effect that choice has, they show varying results depending on the task and role in the process. Their work suggests a methodical approach: first understand the impact of switching models, then look at how that change behaves in different spots, and finally decide where to apply the stronger models for best results.
large language modelsmodel routingworkflowmodel substitutionnumeric frequency countingRMSEsolve-merge-verifytier laddermodel deploymentcalibration
Authors
Renxiang Wang, Jiaming Cui
Abstract
Large multi-call LLM systems pose a scientific problem that query-level routing does not capture: the value of a model depends on where it enters a dependent computation and on the deployment that surrounds that call. Existing routers typically decide \emph{where} to spend a stronger model while treating the benefit of the substitution itself as known. We separate these two decisions through a predicate-action factorization and evaluate it in controlled solve-merge-verify workflows spanning 8-64 solve calls and four three-tier model ladders. The resulting evidence reveals a consistent principle beneath apparently conflicting outcomes. On numeric frequency counting, all-strong reduces RMSE from 4.818 to 1.538 in the Mixed Qwen/GPT ladder, whereas the average Qwen-only ordering reverses. Input-matched interventions further show that the same medium-to-strong action has sharply different value across roles and scales. A semantic task-and-contract shift reverses the Mixed ordering again, while allocation ablations distinguish useful sparse placement from under-coverage and indiscriminate escalation. Together, these results establish model substitution as a deployment-conditioned action rather than a property implied by a tier label, and they provide a practical sequence for large-scale workflow routing: calibrate the action, resolve its role-conditioned effect, and then optimize its placement.