Nearly a year ago, the National Institutes of Health began working on recommendations to improve data reproducibility in pre-clinical research. This initiative began in response to several studies suggesting that over 50 percent of pre-clinical research results were not reproducible.
Yesterday, the NIH released details of an agreement with many scientific journals that establish new rules for publishing research. Once implemented by the journals, researchers will need to adhere to a variety of guidelines regarding data reproducibility and statistical significance. According to the agreement, in the Information for Authors sections of their websites, journals should:
- Clearly define the level of rigor in statistical analyses required for publication and develop methods to check statistical accuracy
- Remove length limits in methods sections to promote transparency in methods reporting
- Ensure experimental and statistical rigor by requiring researchers to disclose:
- The number of biological and technical replicates and the number of times the reported result was observed
- Whether samples were randomized and if experimenters were blind to sample identity
- All statistical methods used to test significance
- A power analysis or other test used to determine sample size
- Any criteria for including or excluding subjects or replicates
- Encourage researchers to adhere to nomenclature and reporting standards
- Ensure data and materials are freely shared with the community and datasets should be deposited in the appropriate database
- Devise a system to deal with scholarly refutation of published articles
Over 70 journals have signed on to the new guidelines including the Journal of Biological Chemistry, which is published by the American Society for Biochemistry and Molecular Biology. Science and Nature published a joint editorial on journals’ role in research reproducibility as well.
Follow the ASBMB Policy Blotter to keep up with science policy issues including the NIH’s work on research reproducibility.