Sector(s): Other
About Opportunity:
The Challenge
Biomedical science is suffering a replication crisis, suggesting that a significant proportion of research resources are wasted and that many results are unreliable. However, this can be countered effectively with simple actions, such as properly accounting for the known risks of bias in in vivo research. Although few studies report the measures taken to reduce these risks, doing so is a stated requirement for many research stakeholders (such as journals, funding bodies and universities) who wish to increase the standards of their research.
Technology
A text-mining tool has been developed which can identify the reporting of measures such as randomisation, sample size calculations, and blinded assessment of outcomes, in research publications. Aggregating this information for a large number of publications allows research output to be monitored, and comparisons – such as changes in practice over time, or between different journals – to be made. The technology is a new method to drive improvement in the standards of biomedical research by providing a rigorous metric for the quality of research publications.
Exemplification Data
The text mining method has been validated by several complementary studies (Macleod et al., 2015), some with samples of more than 2500 publications. Comparisons of the reported risks of bias are made between different time periods, journals of different impact factor, and universities. More recent work has shown the method to have sensitivities of 100%, and specificities >80%, for each of three different reported measures.
The Challenge
Biomedical science is suffering a replication crisis, suggesting that a significant proportion of research resources are wasted and that many results are unreliable. However, this can be countered effectively with simple actions, such as properly accounting for the known risks of bias in in vivo research. Although few studies report the measures taken to reduce these risks, doing so is a stated requirement for many research stakeholders (such as journals, funding bodies and universities) who wish to increase the standards of their research.
Technology
A text-mining tool has been developed which can identify the reporting of measures such as randomisation, sample size calculations, and blinded assessment of outcomes, in research publications. Aggregating this information for a large number of publications allows research output to be monitored, and comparisons – such as changes in practice over time, or between different journals – to be made. The technology is a new method to drive improvement in the standards of biomedical research by providing a rigorous metric for the quality of research publications.
Exemplification Data
The text mining method has been validated by several complementary studies (Macleod et al., 2015), some with samples of more than 2500 publications. Comparisons of the reported risks of bias are made between different time periods, journals of different impact factor, and universities. More recent work has shown the method to have sensitivities of 100%, and specificities >80%, for each of three different reported measures.
Key Benefits:
- High sensitivity and specificity for multiple measures
- Fits within the existing framework of research standards
- Easy to monitor and guide improvements in practice
- Reduces the waste of resources on low quality work
Applications:
IP Status: - Allows research stakeholders to track whether their publications meet required standards.
- Can guide future policies for further improvements in quality.
Development Status: Proof of concept
Publication: M. R. Macleod et al. (2015) PLoS Biol. 13(10) e1002273
The University of Edinburgh is seeking commercial partners to license this technology and/or collaborate on further
Social Plugin