The tools, practices and systems programme's response to Reproducibility and Research Integrity Inquiry

The tools, practices and systems (TPS) programme at the Turing represents a cross-cutting set of initiatives which seek to build open source infrastructure that is accessible to all, and to empower a global, decentralised network of people who connect data with domain experts. 

The TPS programme is submitting the following as part of the Science and Technology Committee’s inquiry into reproducibility and research integrity:

The issues in academia that have led to the reproducibility crisis

• Academic incentives are misaligned with reproducibility and research integrity.
• Novelty prioritised over rigour for research output.
• Open research mandates are insufficient, we need reusability mandates.
• Career progression criteria does not account for reproducible work.
• Fragmentation of initiatives promoting responsible research and innovation, open research, and equality, diversity and inclusion.

Roles of different stakeholders 

• As central stakeholders with the strongest position to change research culture in support of reproducibility, funders should re-evaluate and diversify their assessment criteria.
• Research institutions hiring practices could be better focused on the needs surrounding working reproducibly, including time to replicate findings and training skills.
• Although previous open research policies have focused on individual researchers’ work and skills, it is time to consider the systemic changes that are needed for reproducibility.
• Policies should reward publishers who invest in infrastructure that supports reproducible outputs.
• The TPS programme sees the 'reproducibility crisis' as a significant opportunity for the UK government to be a world-leader in supporting reproducibility and research integrity and tackling a systemic issue in academic research.

Policy suggestions to improve reproducibility and research integrity 

• Align research incentives across equity and inclusion, responsible research and innovation, open research, and reproducibility.
• Embrace the diversity of needs across research domains and encourage skill-sharing across them.
• Carefully consider which research metrics align with incentivising reproducible and reusable research.
• Invest in open infrastructure and make it cost effective to use.

We include at the end of the submission evidence of reproducible research conducted at The Alan Turing Institute and a list of workshop participants who contributed to the evidence captured in this report. 

Production of the submission 

This submission to the Parliamentary Inquiry was developed via an open call workshop with different stakeholders from the Turing community. Participants were invited to reflect on their own views to the inquiry areas, to ensure diverse perspectives on this complex topic were reflected in our response and suggestions. Workshop participants are acknowledged at the end of the document. 

For this response we will focus on three of the five topics outlined in the call for evidence. Specifically: 

1.    The issues in academia that have led to the reproducibility crisis, i.e., its causes.

2.    The role of the following in addressing the reproducibility crisis:

• research funders, including public funding bodies;
• research institutions and groups;
• individual researchers;
• publishers; and
• Governments and the need for a unilateral response / action.

3.    What policies or schemes could have a positive impact on academia’s approach to reproducible research.

Acknowledgements

Malvika Sharan, Tools, Practices and Systems, The Alan Turing Institute
Georgia Aitkenhead, The Alan Turing Institute 
Aida Mehonic, Tools, Practices and Systems, The Alan Turing Institute
Ann Gledson, University of Manchester (Research IT)
Nadia Papamichail, University of Manchester/The Alan Turing Institute
Louise Bowler, Research Engineering Group, The Alan Turing Institute
Martin J. Turner, University of Manchester (Research IT)
Kirstie Whitaker, Tools, Practices and Systems, The Alan Turing Institute