There are a lot of reasons why reproducibility–the ability to replicate results*–is a problem. Some of it has to do with the Decline Effect (and inadequate funding that causes the Decline Effect). There are also basic power of test issues–not running a large enough experiment. While I think funding agencies have largely been asleep at the switch on this, this recent report will force them to confront the issue (pdf; boldface mine):

Until recently, Ain was renowned for a highly prized repository of 18 immortal cancer cell lines, which he developed by harvesting tissue from his patients’ tumors after removal, carefully culturing them to everlasting life in vials. Laboratories around the world relied on the “Kentucky Ain Thyroid,” or KAT lines, both to gain insight into cellular changes in thyroid carcinoma and to screen drugs that might treat the disease, which strikes more than 60,000 Americans each year.

In June 2007, all that changed. Ain attended the annual Endocrine Society meeting in Toronto, where Bryan Haugen, head of the endocrinology division at the University of Colorado School of Medicine, told Ain that several of his most popular cell lines were not actually thyroid cancer. One of Haugen’s researchers discovered that many thyroid cell lines their laboratory stocked and studied were either misidentified or contaminated by other cancer cells. Those included some of Ain’s. They were now hard at work unraveling the mystery.

There was a disaster brewing — it just wasn’t yet official.

Ain was shocked, and justifiably so. Research based on such false cell lines would undermine the understanding of different cancers and possible treatments, and clutter the scientific literature with bogus conclusions.

“At first I thought perhaps their samples were contaminated, not mine,” Ain recalls. “So I undertook systematic thawing and genotyping of all my lines.” He found that 17 of the 18 most frequently shared KAT lines were imposters….

Across different fields of cancer research, up to a third of all cell lines have been identified as imposters. Yet this fact is widely ignored, and the lines continue to be used under their false identities. As recently as 2013, one of Ain’s contaminated lines was used in a paper on thyroid cancer published in the journal Oncogene.

“There are about 10,000 citations every year on false lines — new publications that refer to or rely on papers based on imposter (human cancer) cell lines,” says geneticist Christopher Korch, former director of the University of Colorado’s DNA Sequencing Analysis & Core Facility. “It’s like a huge pyramid of toothpicks precariously and deceptively held together.”

Needless to say, this is a major factor in making reproducibility a problem: if you’re not using the same cell line–or even the same kind of cancer–you’re not repeating the experiment. But, as you might guess, there are real structural impediments in fixing this problem. Can you guess what they are?

…the problem of rampant laboratory contamination is out in the open for all to see, widely acknowledged by the National Institutes of Health (NIH), the National Cancer Institute, major journals and innumerable bench scientists. Yet efforts of concerned scientists have failed to stanch the tide.

“I now give regular lectures about cell line contamination,” says Ain, “and every last person in the audience is shocked and horrified. But most scientists are not willing to test and verify their lines. The NIH doesn’t require it. Very few journals require it. And I can tell you that many scientists are reluctant to disembowel their curriculum vitae, even after they find out a cell line is false. What is an ethical researcher to do?”

Just as in microbiology, you must verify the identity of your materials, otherwise there’s no point in running the experiment.

This needs to be fixed, or it could undermine the NIH’s credibility.

*One can argue that replication and reproduction of results aren’t the same thing, but if you know about that, then you probably didn’t need to read the introduction of this post. So don’t be pedantic.