Statistical Abuse and Teacher Tenure

Last week, a California judge ruled that California’s teacher tenure laws were unconstitutional. In his decision, he cited testimony that one to three percent of teachers are “grossly ineffective.” It turns out that number is bullshit (boldface mine):

It’s made up. Or a “guesstimate,” as David Berliner, the expert witness Treu quoted, explained to me when I called him on Wednesday. It’s not based on any specific data, or any rigorous research about California schools in particular. “I pulled that out of the air,” says Berliner, an emeritus professor of education at Arizona State University. “There’s no data on that. That’s just a ballpark estimate, based on my visiting lots and lots of classrooms.” He also never used the words “grossly ineffective.”

The phrase appears to have been Treu’s shorthand to describe teachers whose students consistently perform poorly on standardized tests. But Berliner is a well-known critic of using student test scores—or “value-added models,” in the parlance of education experts—to measure teaching skills. In part, that’s because research suggests that teachers don’t really control much of how their pupils perform on exams; according to the American Statistical Association, they influence anywhere between 1 percent and 14 percent of the variation in students’ scores. As result, teachers often don’t deliver the same results year after year.

Still, if you look at enough data, there are always a few teachers who consistently underperform on test results. “There’s an occasional teacher who shows up really good a few years in a row,” Berliner said. “There are a few who show up really bad.” And that’s where the now-infamous statistic comes in. During a deposition, Berliner told me, the plaintiffs’ lawyers asked how many teachers deliver low test scores year after year. He didn’t have a hard number, so he said 1 percent to 3 percent, which he thought sounded suitably small….

he doesn’t necessarily believe that low test scores qualify somebody as a bad teacher. They might do other things well in the classroom that don’t show up on an exam, like teach social skills, or inspire their students to love reading or math….

In hundreds of classrooms, I have never seen a ‘grossly ineffective’ teacher,” he told me. “I don’t know anybody who knows what that means.”

Just to review, there were no actual data presented regarding how many poorly performing teachers there are in California. It’s all the more troubling since subject matter test scores and quality of life outcomes are not correlated–and teachers have much stronger effects on the latter, yet they are being judged by scores over which they have very little control.

This is awful law. It is public policy by Magic-8 Ball.

Related: It turns out that none of plaintiffs had bad teachers:

None of the nine named Plaintiffs established that he or she was assigned to an allegedly grossly ineffective teacher, or that he or she faces any immediate risk of future harm, as a result of the challenged statutes. The record contains no evidence that Plaintiffs Elliott, Liss, Campbell or Martinez were ever assigned a grossly ineffective teacher at all. Of the remaining five Plaintiffs, most of the teachers whom they identified as “bad” or “grossly ineffective” were excellent teachers.

A dreadful ruling.

This entry was posted in Education, Statistics. Bookmark the permalink.

One Response to Statistical Abuse and Teacher Tenure

  1. mrtoads says:

    It’s a dreadful ruling if you’re assuming the goal is to improve teaching or to improve the effect of teachers on students. If you assume that the goal is to make teachers subject to employment by whim, making them far more sensitive to ideological pressure (to pick an example purely at random), why then it’s an excellent ruling. History suggests that there is a faction in this country which does not desire that any employee should have any ‘rights’. The controlling faction of the US Supreme Court appears to follow that idea and it would appear that the lower courts are more and more inclined to agree.

Comments are closed.