Design thinking loops converge to local innovation under conditions
When Does Design Thinking Work? A Convergence Theorem for the Iterate-Test Core
Machine Learning
Summary
Design thinking is a cycle of coming up with ideas, making prototypes, and testing them, but it’s not clear when this process actually improves outcomes. The authors represent this cycle as searching through a set of prototypes and show that, under certain conditions, teams will eventually settle on a locally best prototype. However, in practice, limitations like biased tests and limited time often prevent reaching the best outcomes. They also find that testing strategies matter—testing every idea once is usually worse than testing the strongest ideas multiple times.
What this means in practice
- •For product development teams: Design test schedules that repeatedly evaluate top prototypes rather than testing all once to improve innovation outcomes.
- •For software engineering teams: Use screening steps before prototyping to increase chances of developing higher-value software concepts efficiently.
Tested on simulated data.
Authors
Paweł Kuraś, Adrian Michalski, Alicja Gerka, Patryk Organiściak
Abstract
Design thinking is widely taught as an iterative loop of ideation, prototyping, and testing, yet conditions under which such a loop must improve innovation outcomes remain unspecified. Taking problem framing as given, we formalize the iterate-test core as search over a finite prototype space whose utility is learned through noisy evaluations. We prove that the prototype a team believes to be best converges almost surely to a local optimum under five conditions: finite framing, local ideation, non-summable discovery rates, unbiased repeated testing, and retention of tested prototypes. Bounded test bias or criteria drift degrades the guarantee by a bounded amount, and multiple stakeholders yield convergence to a locally Pareto-optimal set. Finite-horizon simulations across three landscape classes (additive valuation, NK rugged landscapes, and clustered modular architectures) show that asymptotic limits are rarely reached in practice. Teams face an optimizer's curse, overestimating their chosen concept by 4-8% and frequently discarding superior prototypes. Testing every prototype once proves to be the worst allocation; optimal performance requires building one to three prototypes per five tests and repeatedly re-testing leaders. Pre-screening concepts yields greater performance gains than tuning testing policies.