In the midst of an article about Uncommon Schools, a non-profit charter school company, we discover this interesting effect of holding back students who fail a grade:
High retention rates can help to boost test scores at charter schools, at least in the short term. Students may do better on tests the second time, and retained students’ scores are dropped from their cohort, so a class of students could improve its test scores over time because the lowest performers have been removed. And sometimes low performers simply leave the charter school when they find out they’re going to be held back.
But it’s good for the kids, right? Well:
Nick Montgomery, a senior research analyst at the Consortium on Chicago School Research, which conducted a long-term study of retention policies, says his team found that students who were held back after elementary school were more likely to drop out later on. In addition, Montgomery says, “students who repeat grades during middle school learn somewhat less than their [low-performing] peers who don’t repeat.”
…But the Chicago research found that simple repetition is not enough — students who are struggling to understand their coursework generally need more intensive interventions, not just a do-over. Retention brings other problems, too. Researchers have written about the psychological impact of separating students from their age group. It can be hard on the retained students, but also on teachers and younger students. They may have to contend with kids who are 15 and 16 years old in middle school and who may have behavioral problems in addition to their academic issues. At the same time, students who are retained are by nature already struggling and more likely to be disengaged with school. Being held back can increase their frustration by moving back the finishing line of graduation and, in the words of the Chicago Consortium researchers, making it seem less “worthwhile to continue.”
But, when not taken into account, retention can alter the evaluation of charter schools:
Every year, local papers in New York City compare the test scores of students in the city’s charter schools to those of students in regular public schools, and charter schools usually come out ahead. But Gill, the Mathematica researcher, notes that “a kid who is taking the test for a second time around is likely to do better than the first time around,” giving charters an advantage in each year’s media contest.
Retention rates also affect the outcome of more sophisticated analyses. Students who are held back are typically a school’s worst performers, but often such students can’t be counted in studies. For the Mathematica study that found KIPP students outperformed regular public-school students, Gill says he and his co-authors had to estimate the performance of students who’d been held back so as not to have missing data.
Sean Reardon, a professor of education at Stanford who specializes in statistical methods, consulted on a study of student achievement in KIPP middle schools in the San Francisco Bay Area. The researchers encountered a problem, however: Many students who started at the schools in fifth grade didn’t reach eighth grade within three years because they’d either moved to another school or repeated a year, making longitudinal measurements difficult. “Studies that don’t deal with this carefully should be suspect,” Reardon says. “They run the risk of producing estimates that make charter schools look better than the other schools.”
Many “high-performing” charter schools have exceptionally high dropout rates. This “improves” their ratings. Meanwhile, many of these students who drop-out return to the public schools who must try to succeed where the charter school failed.
But, remember, educational ‘reform’ is all about the evidence. Or not.