Admittedly, I’m not sure what the alternative would be. But whenever discussions of NIH policy arise, inevitably these two figures by former NIGMS director Jeremy Berg showing a weak correlation between grant funding and paper output are raised.
What I don’t understand is why this is seen as a potential indictment of the wastefulness of ‘big money’ (i.e., multiple grant) labs. If you compare both the preparation process and the review process, grant reviews are far more rigorous than manuscript reviews (as terrifying as that notion is). The money axis would appear to be the ‘better’ axis. In other words, the problem might not be ‘big money’–the big money PIs did gain the approval of other scientists for something far more scarce than a positive review: funding. And there are a lot more backup options for publishing garbage papers than there are for getting garbage proposals funded. The precision of publishing leaves much to be desired.
Yes, it is perverse that grant writing–and approval–are a more precise measure of the perceived (or misperceived) quality of one’s research program than public communication of science (i.e., papers). Nonetheless, it seems this is the more accurate measure of productivity, even though it doesn’t help us determine ‘productivity’*.
*Which of course will also vary field-by-field. Some results are far more expensive per dollar than others.
mikethemadbiologist
I’ve always thought it would be neat to have a sort of “network model” of papers – who each paper is by, cites, and is cited by (possibly even with frequency of citations in each paper as the strength of each link). Then you could apply some sort of fancy networks metrics I know nothing about to calculate how many papers are within 2 steps of a selected pub, how often a given author is cited by papers they’re *not* an author on (to assess how much impact the lab is having beyond citing themselves), etc. You could even do a “six degrees of Kevin Bacon” type thing, like mathematicians do with the Erdos number.
@HCA: many of the visualization tools you want are already available. For example, take a look at http://academic.research.microsoft.com/VisualExplorer , in particular things like coauthor graph and coauthor path.