From tech leaders to educators to government officials, many people now believe that the future of education will be personalized. But a new study by a team of researchers at Columbia University’s Teachers College cast doubt on claims about the potential of personalized learning. The study questions whether personalized learning holds the potential to serve students in the long term.
Scope of the Study
The study, Final Impact Results from the i3 Implementation of Teach to One, was carried out by Douglas D. Ready, Catherine Conn, Shani S. Bretas, and Iris Daruwala with a grant from the U.S. Department of Education. The evaluation had two key strands: a quantitative assessment of the personalized learning initiative Teach to One’s impact on student mathematics performance and a broader qualitative analysis of the personalized learning system. The research team’s most recent report describes results from the quantitative half of the study, which includes data from 36,158 student-level measurements across 209 school-by-year cohorts located within five Teach to One schools in Elizabeth, New Jersey.
Key Findings on Personalized Learning
This study sought to describe the impact of Teach to One: Math (TtO) on student mathematics outcomes over a three-year implementation period in a group of schools in Elizabeth, NJ. While their findings were not entirely negative, the researchers did find troubling inconsistencies. As they explain in the study’s conclusion:
“We found that performance patterns across the five TtO treatment schools were quite inconsistent, and the TtO estimates for each of the three implementation years were statistically non-significant. We found similar results using a measure indicating the combined (averaged) three-year impact and models that excluded a number of relatively higher-achieving control schools.”
The researchers raise specific concerns about whether or not personalized learning is aligned with the expectations found on state tests and whether or not personalized learning does, in fact, serve learners in the long term.
“One result of this differentiation is that TtO and non-TtO students with the same initial test scores likely receive quite different mathematics content during the academic year. Because TtO students may receive content that is below grade-level, their performance on state-mandated, grade-level standardized assessments may lag behind their non-TtO peers who are being exposed to new, grade-level skills over the course of the academic year. This would be the case regardless of whether non-TtO students possess the fundamental skills that new Classrooms staff argue are needed for long-term success in mathematics. In other words, mere exposure to increased levels of grade-level content—although possibly inappropriately advanced—may provide a near-term impact on grade-level assessments. But it may not be the best approach for all students in the longer term.”
The Study’s Noted Limitations
The study’s authors admit that their findings do have several limitations. First, the results come from a single school district staffed by a particular group of teachers. Moreover, while the study was adequate from an analytical perspective, the researchers note, “Our ability to identify statistically significant treatment effects that were modest in size—again, either positive or negative—was somewhat limited by the school sample size.” Finally, they note that the findings of their current study should be taken with a grain of salt given that the TtO model is constantly evolving:
“Its design elements and algorithms are not fixed and can presumably improve over time. As such, it is unclear the extent to which the model examined during this implementation period in Elizabeth, NJ is representative of the TtO model currently being used in schools today. In sum, these analytic complications, which would likely be common to many evaluations of personalized learning models, obviously warrant further consideration.”
Despite the study’s limitations, it does raise legitimate concerns about whether or not some claims on personalized learning have been exaggerated.