Category Archives: Statistics

Maybe This Has Something To Do With The Reproducibility Problem?

Recently, the Atlantic had an article about people who earn a living as clinical trial test subjects. Some ‘methodological notes of interest’ (boldface mine): Intrigued by the promise of an easier way to make money, he enrolled as a guinea … Continue reading

Posted in Drugs, Statistics | Leave a comment

Police Officers Get ‘Teachered’: Accountability Vs. Transparency

A while ago, I contrasted our expectations for teachers versus police officers: It’s weird: one group of heavily unionized public workers is under a constant barrage of claims of being underqualified, while another, with much lower educational standards, gets a … Continue reading

Posted in Statistics, The Rule of Law

P-Hacking Is a Poor Reason For Replication Problems

With the recent release of the results from the Reproducibility Project–an effort to replicate 100 experimental findings in social psychology–there has been a lot of commentary and coverage about the finding that the majority of the studies failed to replicate. … Continue reading

Posted in Philosophy Shit, Statistics | 1 Comment

Reliability and VAM

One of the problems of using value-added measurement (‘VAM’) to assess individual teachers is the imprecision of these estimates. Brad Lindell explains the problem in testimony in a New York case challenging teacher assessment: If the same test-retest reliability from … Continue reading

Posted in Education, Statistics

How You ‘Pre-Treat’ Your Data Really Matters (It’s Not the P-Hacking)

There’s an 538 article “Science Isn’t Broken” that’s winding its way through the tubes of the internet. It has a cool ‘p-hacking’ interactive feature, which is probably why most of the associated commentary has focused on the problem of p-hacking … Continue reading

Posted in Education, Genomics, Statistics

VAM-Based Teacher Evaluation Collides With the Legal System

And if the judge’s questions are any indication, VAM-based teacher evaluation gets hit and dragged a couple hundred yards (boldface mine): Lederman’s suit against state education officials — including John King, the former state education commissioner who is now a … Continue reading

Posted in Education, Statistics | 1 Comment

Understanding the Limitations of Your Data: At-Risk Versus Low-Income

A while ago, I described how decisions about how to treat your data can have significant effects on your results–effects that can be as profound as, if not greater than, p-value tweaking. So a preliminary analysis of the effects of … Continue reading

Posted in Education, Statistics