'The Turing Way' – A handbook for reproducible data science

Introduction

Reproducible research is work that can be independently verified. In practice, it means sharing the data and code that were used to generate published results – yet this is often easier said than done. 'The Turing Way' is a guide to reproducible data science that will support students and academics as they develop their code, with the aim of helping them produce work that will be regarded as gold-standard examples of trustworthy and reusable research.

Project aims

In the ideal case, all published results should be independently verifiable and suitable for other researchers to build upon. For this to happen, the data and code that support the publication need to be made available in an easy-to-use and open format.

Sharing these research outputs means understanding data management, library sciences, software development and continuous integration techniques: skills that are not widely taught or expected of academic researchers and data scientists.

'The Turing Way' is a handbook to support students, their supervisors, funders and journal editors in ensuring that reproducible data science is 'too easy not to do'. It will include training material on version control, analysis testing and open and transparent communication with future users, and build on Turing Institute case studies and workshops.

Applications

'The Turing Way' will support everybody involved in data science research: the developers of the code (research engineers, postdocs and doctoral students), their supervisors and the business team members who coordinate these projects. The format will be easy for the reader to dip in and out of, depending on their level of experience in the various topics. The project will help to answer questions that researchers don't always ask: "How do I ensure that my code's existing functionality doesn't change as I extend the codebase?", "How do I make my project easy for someone else to run?", and many more.

Senior team members – Turing fellows, program directors and managers – will be catered for with key points tailored towards managing reproducible research projects highlighted for each topic covered. The project will build and curate checklists for what can be done to ensure all project outputs are reproducible. A chapter on Binder will be of interest to supervisors who want to regularly review their students' code, and will include the technical details of how to set up a BinderHub that will be useful for research software engineers.

Recent updates

March 2021

Next Book Dash, 17–21 May 2021

We are inviting applications to participate in the next Book Dash event taking place from 17–21 May 2021. Applications can be submitted via this Google form. The deadline for submission is 15 April 2021.

The Book Dash events bring together participants to work on The Turing Way book synchronously either in person or online. The upcoming book dash will be organised for online participation using the format we developed last year (see an overview here). You can find more details in the application form, which will take up to 30 minutes to complete. If you are not sure if your idea fits the event, you are welcome to contact Malvika Sharan to discuss it further by emailing [email protected].

Overview of resources

GitHub repository
The Turing Way book

The Turing Way guides

Over the past year The Turing Way has expanded to five community-produced guides:

* Guide to Reproducible Research
* Guide to Ethical Research
* Guide to Project Design
* Guide for Communication
* Guide for Collaboration

By December 2020, the project had over 250 GitHub contributors and nearly 2000 subscribers to our communication platforms.

It currently has 35 chapters, 134 subchapters and several supporting resources within the five guides and a community handbook.

Illustrations

Since 2019, we have developed useful illustrations collaboratively with The Turing Way Book Dash participants using the scribing service of Scriberia. These have been shared for community use on Zenodo and have been downloaded nearly 4000 times to use in different Open Source materials. A new set of illustrations was released in December 2020 that can be reused under CC-BY 4.0 license with the appropriate citation: “This image was created by Scriberia for The Turing Way community and is used under a CC-BY licence. Zenodo. http://doi.org/10.5281/zenodo.4323154”.

Mentorship support

The Turing Way members offer guidance, review and mentoring for facilitating contributions by new members.

We have also collaborated with Open Life Science (OLS for Turing) to support our members to receive mentoring and training on applying open research principles in the work within The Turing Way.

In 2020, 12 members from across 6 projects in the Turing and The Turing Way community successfully graduated from the program’s second cohort. Details here.

To support this collaboration in 2021, the Turing Online Training Grant has been offered to the Open Life Science in association with The Turing Way.

Connect

Email
Slack workspace
Twitter Channel
Newsletters

Learn more

Impact Report 2019-2020
YouTube Videos
Book Dash Reports
Quarterly reports 2020

Selected reports and articles

* An Emerging Technology Charter for London (2020). Access here
* Reproducibility of scientific results in the EU : scoping report (2020). Publications Office of the European Union. Access here
* Innovation Scholars: Data Science Training in Health and Bioscience – UKRI. (2021, January 08). Access here
* Report by FREYA project “Connected Open Identifiers for Discovery, Access and Use of Research Resources”: [Access here]
* Top tips for making the most out of Binder, blog post by Emma Karoune, Sarah Gibson, Martina Vilas, and Sophia Batchelor, 2020, Software Sustainability Institute.
* Training material “Creating JupyterBook with The Turing Way” at JupyterCon 2020, Martina Vilas, Sarah Gibson and Malvika Sharan, 5-9 October 2020: https://github.com/martinagvilas/tutorial-jupyterbook-with-turing-way
* Collaborative work in a pandemic - The Turing Way Bookdash November 2020, blog post by Emma Karoune, 2020.
* CW20 speed blog: Bootstrapping a development team during the time of crisis, Raniere Silva, Malvika Sharan, Colin Sauze, Yo Yehudi, Claire Wyatt, 2020, Software Sustainability Institite. (Available as a chapter in The Turing Way: Managing a new community and a team.
* Experience of The Turing Way Book Dash as a first-time participant, blog post by Arielle Bennett-Lovell, 2020, CSCCE forum.
* The Turing Way: An open source resource promoting best practice for reproducible research, blog post by Becky Arnold, 2019, Software Sustainability Institute.