Reproducibility of Published Research
|Reproducibility of published research has recently become a hot topic. Notably, in 2012, a group of scientists from Amgen and Bayer, led by Glen Begley, tried to reproduce 53 high-impact oncology publications and found that only 7 were reproducible (a horrific rate of 11%). As you can imagine, this impacts every group involved in the research path, from funding agencies, such as the NIH, to academic researchers who cite these papers for their own research, to the biotech and drug companies, who pursue new therapies based on published research. It affects everyone. It is such a concern that in January of 2014, in the journal Nature, NIH Director Francis Collins and Principal Deputy Director Lawrence Tabak commented on the importance of reproducibility, noting several ways in which the agency hopes to fix the problem. Just think about the research that you do now and consider that possibly only 1 in 10 of the papers that you use to base your research on is reproducible.||
One would hope reproducibility is not as rare as finding a two-legged race horse...
|The attention to reproducibility is not really new. One Stanford professor, Dr. John Ioannidis, has been studying it for over 20 years. His 2005 paper titled “Why Most Published Research Findings are False” caused quite a commotion in the scientific community. He points out the single most important cause for false findings: bias, either conscious, subconscious, or unconscious. As an example, he often starts his lectures describing how 40 out of 50 randomly selected cookbook ingredients have been found to cause cancer. His purpose is certainly not to discredit all science, rather the opposite; he points out that self-correction is part of the scientific process and he identifies ways that science can improve. He specifically recommends trying to get independent investigators to replicate findings, pooling resources to create consortiums for studies, making raw data more publicly available, and reducing inefficiencies in research, including revising the reward system for research.||
Dr. John Ioannidis speaking at the
NIH in 2012.
|Issues with reproducibility have now led to the establishment of the Cancer Biology: Reproducibility Initiative with the support of Science Exchange, PLOS, Figshare, and Mendeley to provide a mechanism for scientists to independently replicate findings and be rewarded for doing so. Earlier this year, the initiative took a significant step forward when it received a $1.3 million grant from the Laura and John Arnold Foundation to validate 50 recent landmark cancer biology studies.|
|BioLegend is happy to announce that we have formally agreed to provide Science Exchange $100,000 in research reagents, such as flow cytometry antibodies, functional antibodies, ELISA Kits, and recombinant proteins, in order to support the Reproducibility Initiative. Read the full press release here.
We certainly don’t believe this is just a witch hunt to point fingers at any individuals. This is another step that science needs to take in order to affirm that what we do is truly scientific and that research findings will lead to real progress and benefit everyone. What are other ways we can improve reproducibility?
To learn more about Glen Begley's findings visit:
|Contributed by Dzung Nguyen, PhD.|