Once Again, Papers Are Professional Currency

At least in biology. And nothing will change until funders decide things should change.

Mathematician Timothy Gowers writes about peer review (boldface mine):

Defences of formal peer review tend to focus on three functions it serves. The first is that it is supposed to ensure reliability: if you read something in the peer-reviewed literature, you can have some confidence that it is correct. This confidence may fall short of certainty, but at least you know that experts have looked at the paper and not found it ­obviously flawed.

The second is a bit like the function of film reviews. We do not want to endure a large number of bad films in order to catch the occasional good one, so we leave that to film critics, who save us time by identifying the good ones for us. Similarly, a vast amount of academic literature is being produced all the time, most of it not deserving of our attention, and the peer-review system saves us time by selecting the most important articles. It also enables us to make quick judgements about the work of other academics: instead of actually reading the work, we can simply look at where it has been published.

The third function is providing feedback. If you submit a serious paper to a serious journal, then whether or not it is accepted, it has at least been read, and if you are lucky you receive valuable advice about how to improve it.

Gowers proceeds to knock these claims down rather convincingly. But what he neglects is the role that peer-reviewed papers play as professional currency. As some asshole with a blog put it:

I’m not convinced that the de facto purpose of publication is (note: is, not should be) is to communicate results to other scientists. Realistically, when someone publishes a paper, is she thinking, “Now all my colleagues can hear about my research!”? Or is she really thinking, “I’m that much closer to getting my R01 grant renewed”? Obviously, papers are useful, especially for older projects (or those belonging to dead scientists…). But, if you’re like me, you probably have a stack of unread papers, or a folder of pdfs (or both) that you never get around to reading. It’s not a very good system to disseminating information when you get right down to it.

But where publication, as it exists today, is critical is in the funding process. Most funding agencies will only view ‘legitimate citations’, both in terms of justifying your work as well as establishing productivity, as those papers that are published in peer review journals (until recently, even papers in press didn’t count–they had to be published). If we want to move towards a better publishing model–and I would argue that should include considering new formats–then we need to get funders on board with various proposals, or else those proposals will not get very far.

Or as same said asshole snarked:

…do you know the difference between a couple of papers in BiorXiv and in journals?

$250,000 per year in direct costs from NIH, and somewhere between $75,000 and $125,000 in indirect costs.

And DrugMonkey made a related point (boldface mine):

This whole thing is getting ridiculous. I don’t have the unfettered freedom to decide where to publish my stuff and it most certainly is an outcome of the funding agency, in my case the NIH.

Here are the truths that we hold to be self-evident at present time. The more respected the journal in which we publish our work, the better the funding agency “likes” it. This encompasses the whole process from initial peer review of the grant applications, to selection for funding (sometimes via exception pay) to the ongoing review of program officers. It extends not just from the present award, but to any future awards I might be seeking to land.

Where I publish matters to them. They make it emphatically clear in ever-so-many-ways that the more prestigious the journal (which generally means higher IF, but not exclusively this), the better my chances of being continuously funded.

And DrugMonkey was discussed peer-reviewed open access papers (which are often thought of as less prestigious, especially compared to the glamour magz), not non-peer reviewed manuscripts.

If you want to change (or eliminate) the peer review system, you have to get the funders to buy in. Until then, you’re asking scientists to take huge, probably unsustainable, career hits.

This entry was posted in Publishing. Bookmark the permalink.

3 Responses to Once Again, Papers Are Professional Currency

  1. drugmonkey says:

    NIH permits citation of preprints in grant applications now. This changes things somewhat.

  2. Only if the reviewers accept preprints as valid.

  3. Potnia Theron says:

    But also.. one of the reasons his two examples of pre-print review worked is that Perelman and Blum are 1) known entities 2) working on Important Problems (within their field). So yes, if people are willing to read and comment on the preprints, it would work, and work well.

    But what about Josephine Blow, grad student or early career? She might be doing something important, but doesn’t have the visibility. The feedback might matter very much to her. I think you can argue either way about the plight of non-shiny young people in non-cutting-edge fields.

Comments are closed.