Over at Unlearning Economics, there’s a very interesting post about the importance of prediction versus assumptions for theory–a response to economist Milton Friedman’s claim that theories should ultimately be judged by their predictive value. There’s a lot in the post, but two points were especially interesting (boldface mine):

The second problem with a ‘pure prediction’ approach to modelling is that, at any time, different theories or systems might exhibit the same behaviour, despite different underlying mechanics. That is: two different models might make the same predictions, and Friedman’s methodology has no way of dealing with this.

There are two obvious examples of this in economics. The first is the DSGE models used by central banks and economists during the ‘Great Moderation‘ predicted stable behaviour, and the economy exhibited this. However, Steve Keen’s Minsky Model also exhibits relative stability for a period, before being followed by a crash. Before the crash took place, there would have been no way of knowing which model was correct, except by looking at internal mechanics.

Another example is the Efficient Market Hypothesis. This predicts that it is hard to ‘beat the market’ – a prediction that, due to its obvious truth, partially explains the theory’s staying power. However, other theories also predict that the market will be hard to beat, either for different reasons or a combination of reasons, including some similar to those in the EMH. Again, we must do something that is anathema to Friedman: look at what is going on under the bonnet to understand which theory is correct.

This is similar to a point I made about competing theories of baseball catching: under most situations the ‘outfielder as calculus engine’ model does reasonably well, though not as well as an ‘outfielder following simple mechanistic rules’ model. But there are minor discrepancies that become huge flaws in the ‘calculus engine model’, such as how players would react to shallow, deep line drives.

But there’s a more basic problem (boldface mine):

The final problem, less general but important, is that certain assumptions can preclude the study of certain areas. If I suggested a model of planetary collision that had one planet, you would rightly reject the model outright. Similarly, in a world with perfect information, the function of many services that rely on knowledge – data entry, lawyers and financial advisors, for example – is nullified. There is actually good reason to believe a frictionless world such as the one at the core of neoclassicism leaves many firms and entrepreneurs obsolete. Hence, we must be careful about the possibility of certain assumptions invalidating the area we are studying.

I think one reason biology has made signifcant progress is that there isn’t a unifying theory–different disciplines have separate domains of explanation: crystallography and population biology don’t exclude each other (although this isn’t to say that the results from one discipline can’t help us make sense of the other discipline’s results; they surely do).

Something to think about.