False Positives, False Negatives, and Validation in Economics

As I always note before ripping into economics, we need better economics, not simply to disparage the entire field. Economics does matter. Anyway…

Peter Dorman notes this about the refusal of several physicists to co-author a recent paper from the OPERA project that might simply be a measurement error (boldface mine):

The answer is that, with such an extraordinary anomaly, there is a risk of error. Mismeasuring the distance within the apparatus by 12 meters, for instance, would reverse the results. Above all—and this is why economists should be interested—a physicist would suffer a huge, possibly irreparable blow to his or her career by being attached to a claim that is later found to be wrong. Type I error (false positives) are taken very seriously. The logic of this extreme asymmetry, so much weight on Type I, so much less on Type II, is explained in this earlier EconoSpeak post.

It’s rather different in economics, isn’t it? If someone shows you have made a false claim in a published article, you can write a gracious response thanking the critic and go on. More likely, you will double down and spin out more studies defending your original argument. Either way, if you’re wrong it’s no big deal. Lots of the top economists in the professional firmament have been wrong at one time or another (or even all the time), and it hasn’t set them back. Meanwhile, physics evolves over time toward ever-closer approximation of the real universe, while economics accumulates error along with insight.

I think part of the explanation for this has to do with an unwillingness (or lack of perceived need) to pound your hypothesis or model with data. And pound it hard. I can’t help but think that part of the reason for the lack of concern with type I errors has to do with a belief that modelling-based deduction by itself constitutes sufficient inquiry.

Which, I think, is very odd to most physical scientists.

This entry was posted in Economics, Hypotheses. Bookmark the permalink.

2 Responses to False Positives, False Negatives, and Validation in Economics

  1. Pingback: Links 10/18/11 | Mike the Mad Biologist

  2. Pingback: Links 10/18/11 | Mike the Mad Biologist

Comments are closed.