A Large-Scale Observational Study on Obtaining Lightweight, Randomized Weekly Student Feedback

Student feedback on their course experience provides valuable input to course instructors to reflect on their instruction and make adjustments to their teaching. To address the inefficiencies of conventional student evaluation mechanisms in obtaining both thoughtful and frequent feedback from students, Kim et al. (\cite{kim2023high}, L@S'23) recently proposed the following simple and lightweight course feedback mechanism: survey each student on random weeks exactly twice per term. Named \emph{High-Resolution Course Feedback (HRCF)}, this method has been shown to elicit specific and timely feedback that instructors found helpful in understanding their students and making instructional adjustments, without imposing excessive feedback burden on the students \cite{kim2023high}. An important question, however, remains unanswered: does this simple method translate into measurement improvements in \emph{students' actual course experiences?} To answer this question, we present a large-scale observational study of HRCF use across 103 course offerings (24,216 total enrollment) over four years between Fall 2021 and Fall 2025), spanning 42 unique computer science courses at an R1 institution. We analyzed the end-of-term student evaluation data for these courses from Fall 2018 to Fall 2025. Through a regression analysis on the average student ratings on 4 different student evaluation items, we find that first-time use of HRCF does not have a measurable effect on average student ratings. However, among small- to medium-enrollment course offerings (<250) that continued using HRCF, each additional term of use is associated with average rating increases of $0.045\sim0.048$ points for learning-grain related student evaluation items. We observe no significant effects for large-enrollment classes ($\geq250$) or for average ratings of instructional quality and organizational quality of courses. These results provide evidence that repeated HRCF use does have a positive association with improved student evaluations of their learning experience, but that the current design of HRCF needs to be enhanced to have measurable impact on broader student perceptions of quality of instruction and course organization. We analyze these results in great detail and discuss what they imply for enhancing HRCF.