Human learning model parameters recoverable only with designed practice data
Identifiability of a dissipative knowledge-dynamics model: exact recovery under designed excitation, degeneration on observational data
Machine Learning
Summary
Learning can be thought of as a process where knowledge builds up, fades, and spreads between related ideas. The authors created a math model that tries to capture this process and studied when you can figure out the model’s settings just by looking at learning data. They found that if the learning experiences are carefully planned, you can exactly recover these settings. But with ordinary, real-world data, you can’t reliably find them, meaning some results about learning difficulty might be misleading. This shows when a learning model is trustworthy and when it isn’t.
What this means in practice
- •For online learning platform developers: Design practice schedules that guarantee accurate recovery of learning dynamics model parameters to better tailor student feedback and interventions.
- •For educational data analysts: Use the authors’ validation protocol to vet interpretability claims from learning models applied on observational data and avoid misleading conclusions about learning behaviors.
Authors
Arman Kostanian, Armen Beklaryan
Abstract
Human learning is a dissipative dynamical process: mastery accumulates through practice, decays through forgetting, and propagates across interdependent concepts. We model it as a nonlinear dissipative system of ordinary differential equations whose parameters are mechanistically meaningful (a concept-transfer matrix encoding prerequisite coupling, per-concept forgetting rates, and a saturating practice-response gain), and we study when those parameters can actually be recovered from data. We prove a structural identifiability theorem for the associated inverse problem under explicit excitation conditions, with constructive closed-form recovery for the two-concept case, together with monotonicity, robustness and L-stability results. We derive a semi-implicit L-stable scheme for the dissipative subsystem and a batched solver numerically equivalent to the per-trajectory formulation (bit-exact predictions, gradients to $10^{-10}$) yet two orders of magnitude faster, making estimation feasible on cohorts of $10^5$ learners. The empirical study is two-sided. Under the theorem's excitation conditions, synthetic recovery is exact: parameters to machine precision, prerequisite structure at $F_1 = 1.0$. On large observational benchmarks it is not. An apparently strong recovery, with forgetting rates correlating with topic difficulty at Spearman $ρ= 0.83$, is refuted by four independent controls: it survives destroying the temporal order of the data, is matched by a classical Bayesian baseline, and is unaffected by removing real timestamps. We trace this to the stationary structure of the model and show that it is the degeneration the theorem predicts in the absence of designed excitation. The result delineates a sharp boundary between identifiable and unidentifiable regimes and yields a validation protocol for interpretability claims.