And if the judge’s questions are any indication, VAM-based teacher evaluation gets hit and dragged a couple hundred yards (boldface mine):
Lederman’s suit against state education officials — including John King, the former state education commissioner who is now a top adviser to U.S. Education Secretary Arne Duncan — challenges the rationality of the VAM model, and it alleges that the New York State Growth Measures “actually punishes excellence in education through a statistical black box which no rational educator or fact finder could see as fair, accurate or reliable.”
Here’s what happened to Lederman: In 2012-13, 68.75 percent of her New York students met or exceeded state standards in both English and math. She was labeled “effective” that year. In 2013-2014, her students’ test results were very similar, but she was rated “ineffective.” Meanwhile, her district superintendent, Thomas Dolan, declared that Lederman — whose students received standardized math and English Language Arts test scores consistently higher than the state average — has a “flawless record.”
…On Aug. 12, New York Supreme Court Justice Roger McDonough presided over a hearing in the case — and he was not amused with the state’s case….
“Did her students learn nothing?” Justice McDonough asked. “How could it be that she went from 14 out of 20 points to 1 out of 20 points in one year?” He noted that the students’ scores were quite good and not that different from the year before.
Back behind the bell curve Ms. Galligan ran. As she tried to explain once again, the judge said, “Therein lies the imprecise nature of this measure.”
Lederman was not the only teacher in the school to get a poor score. In 2014, 21 percent of the staff at E.M. Baker School received a score of “ineffective,” 21 percent “developing” and 57 percent were “effective.” Just the year before, not one teacher received an “ineffective” score…
“There is nothing in the law that requires a bell curve,” [plantiff attorney Lederman] argued. He explained that a bell curve with its forced failures violates that law that requires that every teacher must be able to get all scores. Not only did he want the court to set aside his wife’s score, he wanted the court to “declare the measure an abuse of discretion” because “the State Education Department does not get a pass on unreasonable and irrational actions.”
There are all sorts of statistical problems with evaluating individual teachers using VAM, as we’ve detailed before. But one of its many weaknesses is that teachers who teach ‘extreme’ students (very high or low performing students) get screwed since the statistical methods can fall apart when there is a maximum and minimum score and the teachers’ students are near the minimum or the maximum. To be blunt, poorly-performing students don’t have advocates, but wealthy students–and teachers from those schools do.
This case could have a seismic effect on education ‘reform.’