More Teacher Evaluation Insanity

Education reformers style themselves as problem solvers who get things done. The reality, however, is that they’re grade A fuckups (boldface mine):

A single missing suffix among thousands of lines of programming code led a public school teacher in Washington, D.C., to be erroneously fired for incompetence, three teachers to miss out on $15,000 bonuses and 40 others to receive inaccurate job evaluations…

The problem in D.C. stemmed from “a very small typo” inserted into complex programming code during an upgrade earlier this year, said Barbara Devaney, chief operating officer of Mathematica Policy Research, the private firm that holds the contract to calculate value-added scores for the district.

Devaney said the firm employs stringent quality control, which in this case included 40 hours of meetings to review the updated model and an analysis by independent programmers paid to comb through the code line by line. Yet no one noticed the missing suffix until yet another routine quality review took place this November — after the district had already distributed bonuses, layoff notices and evaluation scores based on the value-added data for the 2012-13 school year, Devaney said…

In all, the error affected 44 teachers in D.C. — about 10 percent of those who receive value-added scores based on their students’ standardized tests. Half were rated higher than they should have been and half were rated lower, Kamras said…

But some critics noted that it may be impossible for the district to “hold harmless” all teachers affected by the error, as Kamras intends. A study released earlier this year found that getting a poor rating prompted many teachers to leave the district or quit the profession, even though they were not fired. It’s unclear whether any of the affected teachers may have altered their career plans after receiving scores that were lower than they actually deserved.

Garbage in, garbage out, I guess. Shouldn’t treat people like garbage though.

This entry was posted in Education, Statistics. Bookmark the permalink.

3 Responses to More Teacher Evaluation Insanity

  1. evodevo says:

    They don’t care – older teachers quit, and now they are free to hire newbies at much reduced salaries. That’s the ONLY criterion for success in their demented neolib books.

  2. grahamsw says:

    This is an argument against doing evaluation badly, not against evaluating. Yes, it’s terrible for the people who got shafted by this, but it doesn’t change my default answer to all complaints about teacher evaluation: we evaluate students all the time, that’s how we tell whether we’re doing our job. We can all have problems with standardized, automated, tests. Our evaluation at the individual level may be more accurate (may, Kahneman and all should make us question that), but for larger comparisons and policy making, standardized tests of some sort are essential. (You want more resources for your school? Prove that you need them without testing students.)

    Teachers are not fungible. Testing should be used primarily to identify training and support needs, as it should be for students, but it should be done. There will be mistakes – as there will be in any complex system. That doesn’t mean you don’t try.

  3. Pingback: How do you define success for a calculus MOOC? | mathbabe

Comments are closed.