Augmenting Knowledge Tracing With Self-Regulated Learning Indicators for Reliable Skill-Mastery Estimation

Many edtech systems use a knowledge-mastery threshold to decide when learners can stop practising a skill and move on to new con- tent. However, learners can cross this threshold even when their recent traces show weak self-regulatory behaviors, such as rapid guessing or getting answers right only after repeated attempts. This raises concern that these learners are likely to struggle on subsequent work once practice is stopped, even though the sys- tem has already advanced them. To address this decision risk, our study proposes an approach that augments standard knowledge tracing (KT) mastery predictions with trace-based self-regulated learning (SRL) indicators, using ASSISTments Skill Builder 2009– 2010 data logs. Specifically, we adopt a trace-based SRL framework and define weak regulation at the mastery boundary as low-quality engagement around the moment when the system first declares a skill “mastered”, operationalised via behavioural indicators de- rived from learners’ recent practice traces. We then test whether KT mastery crossings that co-occur with weak-regulation indicators predict less stable performance after the system ends practice on the skill, including higher rates of later performance drops and slower recovery after errors. Our results show that these “KT-mastered but weakly regulated” cases are common across skills and consis- tently associated with poorer subsequent outcomes than other KT- mastered cases. Together, the findings motivate adding lightweight SRL checks at the mastery boundary to improve the reliability of mastery based progression decisions, especially in learning at scale settings where support resources are limited.