Is NIH Purchasing Risk-Free Science?

Drugmonkey, building on a comment left at his blog, claims the following (boldface mine):

This comment from dsks absolutely nails it to the wall.

The NIH is supposed to be taking on a major component of the risk in scientific research by playing the role of investor; instead, it seems to operates more as a consumer, treating projects like products to be purchased only when complete and deemed sufficiently impactful. In addition to implicitly encouraging investigators to flout rules like that above, this shifts most of the risk onto the shoulders of investigator, who must use her existing funds to spin the roulette wheel and hope that the projects her lab is engaged in will be both successful and yield interesting answers. If she strikes it lucky, there’s a chances of recouping the cost from the NIH. However, if the project is unsuccessful, or successful but produces one of the many not-so-pizzazz-wow answers, the PI’s investment is lost, and at a potentially considerable cost to her career if she’s a new investigator.

…this is absolutely the right way to look at the ever growing obligation for highly specific Preliminary Data to support any successful grant application. Also the way to look at a study section culture that is motivated in large part by perceived “riskiness”…

NIH isn’t investing in risky science. It is purchasing science once it looks like most of the real risk has been avoided.

I think this is confusing cause and effect (and, no, we are starting a philosophical debate on the true nature of causality. You want to do that, go start your own blog). What I mean is that I don’t think this is NIH’s goal at all. Instead, this stems from a very simple dynamic: for every grant that is funded, two to five ‘good’ proposals do not get funded*. In other words, reviewers have to come up with reasons to deny funding to perfectly good proposals. In that environment, things like ‘riskiness’ and Prelimary Data become much more important–arguably more important than they should be.

While the result is the same as Drugmonkey describes (risky science is less likely to be funded), I think the causal mechanism is far more banal: reviewers are trying to distinguish among proposals. One thing that could be done would be to downweight Preliminary Data (though whether reviewers could be trained to do that in terms of their overall impressions is unclear). This, however, could ‘bunch up’ more proposals (i.e., more would have ‘good’ scores), placing more power in NIH’s hands in terms of what should be funded.

Given underlying budget constraints, I can’t see this situation improving.
*The upper bound depends on how you want to define good, as well as the funding rate of the panel.

This entry was posted in Funding, NIH. Bookmark the permalink.