In praise of research engineering

A personal story from a departing Senior Research Data Scientist

Wednesday 26 Jun 2019

Filed under


This is a personal retrospective on my experience at the Turing's Research Engineering Group (REG). Since I decided to leave for a more traditional academic position, I have been asked why I took this job in the first place, and what went wrong with it. TL;DR: nothing went wrong! For me, working in research engineering has been incredibly enriching. Here’s why.

Where I came from

There is history to this story, not surprisingly.

Digital humanities

After degrees in computer science, history, archives and information studies, I ended up doing a PhD in digital humanities at EPFL in Switzerland. There is no research engineering – by that name, at least – within the digital humanities, even though a lot of data and software engineering is going on. Entire groups are increasingly being set up to essentially do this job. For example, the Digital Humanities Lab at the KNAW Humanities Cluster in Amsterdam and the King's Digital Lab in London. Yet this digital humanities activity is somewhat divorced from the research software engineering community, which has its origins in the physical sciences. Digital humanities’ ‘research engineering’ currently entails a focus on digitisation, data modelling and web development, while research engineers from, say, physics, traditionally focus more on high-performance computing and the efficient implementation of algorithms.

Digital humanities?

As I approached the end of my PhD, I was not convinced that I should go for an academic career. Instead, my focus was on gearing myself up for industry research with an underpinning in data science. That would be quite a leap, given my CV.

For sure, my maths and software-development skills, intended as a step up from coding, were not industry-ready. Nor were my so-called ‘soft skills’, such as teamwork and leadership. Not unusual for recently-graduated PhD students. It is also relatively common to be unsure of whether to pursue a career in academia or in industry after a PhD. This is where research engineering can play a role.

An alternative to a postdoc or a career in and of itself?

In a nutshell, my reasoning went as follows: I wanted to maximise my professional options by constantly raising the technical bar, joining a group where I had plenty to learn from my colleagues and from the projects I would work on. I also wished to remain close to research, by exploring new areas and yet keep a strong focus on digital humanities. Moving to a thriving environment such as the Turing in London, where an incredible amount of opportunities can materialise in a short time span, would also push me to clarify which direction to take (industry or academia).

A postdoc would have meant less broad technical development, and it would have provided for more focus on a given topic of research: too unbalanced towards academia for what I wanted at the time. An immediate shift to industry would have meant stepping out of academia, and doing so before my profile was sufficiently developed to go for really compelling opportunities. At the crucial juncture of the end of a PhD, research engineering allows you to explore and gear up for both industry and academia, and it might even become your career choice itself. Thumbs up for research engineering!

What I learned

Given my backstory, how did it actually go?

Curiosity and service

Science is a community effort, and academia is primarily curiosity-driven. This is good, as intellectual curiosity and freedom are key to innovation. There are several ways to contribute, but some, such as developing software or research infrastructure, are notoriously poorly recognised as contributions compared with, say, well-cited papers. Working in research engineering has allowed me to pursue curiosity-driven research much more than I was expecting: as part of a rapidly growing group and institute, it has been possible to follow my own interests to a significant degree.

The way this was done, crucially, was still service-driven: my metric for success was not primarily papers and citations – though these were always welcome and encouraged – but alternative contributions to research, such as data, software, infrastructure and community development. This structure actually allows one to do these things well: a crucial aspect, I believe, of calling what we do ‘engineering’. The net personal gain is that of mixing the best of both worlds: working on interesting projects in my area of choice (curiosity), while developing my technical skills building research tools (service).

Skills matter

The skillset of the research software engineer is becoming a necessity for researchers generally. The days of dirty, unpublished scripts are, hopefully, numbered. Several communities are rapidly realising the importance of open and reproducible research, and taking steps to foster it, so researchers need to skill-up to meet the challenge. Most typically, this entails developing data and code to a higher standard, for example by adopting practices common in industry such as modularity, versioning and test-driven development. Research engineering practitioners commonly adopt and strive for industry standards, while keeping the flexibility required in research. Research engineering offers a great opportunity to develop in this sense. In my case, I made more commits [committed code revisions] in my year and a half with the Turing than during all my previous active years combined.

Working as a team

Another aspect of research which is changing, from my humanities viewpoint, is the scale and complexity of collaborations. While at the Turing, I have been working in the Living with Machines project with its team of 20 to 25 people. This is very large for the humanities – even the digital humanities – where the median project-team size is probably still one.

Larger, more complex projects require working not just in a team, but as a team. How to go about this is not immediately obvious, especially in academia: lessons and methods can be borrowed and adapted from industry, but this requires time and space for experimentation. Research engineering, I found out, is already imbued with many good ways to go about collaborating to develop research software and infrastructure: from using the right tools, such as GitHub and Slack, to meta approaches – I now know the principles of agile development by heart.

‘Soft skills’ matter even more

Collaboration is more about people skills than tools and approaches to go about it. Academia unfortunately does not always shine with respect to mentorship and teamwork. I would argue this is due, at least in part, to the fact that in the academic environment there is little opportunity to go about developing and nurturing these skills.

While working at The Turing, I actually got the opportunity to develop work plans over both long and short time spans and to interact with clients, stakeholders and colleagues. Playing the institutional game was also important to get exposure to the daily workings of organisations – also an important skill in industry.

Yet the best part, at least for me, is that I actually got to ask myself how best I could help someone else succeed professionally, as a lead or line manager. PhD-student to master's-student, or postdoc to PhD-student, relationships are rarely framed in this way, even though they ought to be. In research engineering I found a community of practice and of service where these values are nurtured.

But, you’re a data scientist, not a research engineer. Oh, really?

An addendum to the above relates to the actual job title. The Turing realised, I believe correctly, that the scope and goals of research engineering are broadening. Data science, or the principled practice of going about analysing data to gain insights, is increasingly part of the research process in most disciplines and fields. As a consequence, research engineering is gradually including more and more of data science as the area matures and consolidates. Indeed, in industry, data science teams work closely with IT, analytics and product teams.

In the words of James Hetherington, director of research engineering at the Turing, data science and research software engineering form a continuous spectrum, and this is why the Research Engineering Group at the Institute has a unified job description for both data scientist and research engineer (and, incidentally, everyone in the group can pick the title they prefer!). I picked data scientist, but I’m always happy to say that I work in research engineering.

The future

This experience has left me wanting more, and with some clear ideas to develop.

Shape your tools before they shape you

Traditionally, technical work in the humanities has been separated from research. It may be that no other scholarly area has such a wide separation between researchers and technical support (where technical support exists at all). This tradition dates back centuries, if we include so-called ancillary disciplines such as archival and library studies, palaeography, and diplomatics. While this makes sense from a professional and practical point of view, it is unfortunate and limiting from the perspective of collaborating and engaging in intellectual exchanges.

Data science skills are now a necessity for digital humanities researchers, just as some domain knowledge is necessary for a data scientist engaging in a digital humanities project. It is very difficult to engage in interdisciplinary collaboration without such intellectual bridges. Similarly, as we grow our digital assets in support of digital humanities research, we are increasingly required to understand and shape them first. Research engineering can offer the space to develop and disseminate these skills, while at the same time building the infrastructure we increasingly need.

Research engineering for the digital humanities

Research engineering is going to increase in scope and importance, and the value of developing solid research software and infrastructure goes hand in hand with the spread of data science in research. As for most other fields, I believe this should be embraced in the digital humanities too, by developing our own approach to research engineering. (Even more broadly, research engineering might just be the still missing link between digital humanities research and the GLAM sector – Galleries, Libraries, Archives, Museums). Pressing challenges abound: engineering and infrastructure for digitisation, (meta)data and analysis; development of standards and best practices; training and community-building, to name a few.

The future will hopefully deliver an expansion and consolidation of the various digital humanities labs which have started to open, both in the direction of doing research and of developing research data, software and infrastructure. In due course, this digital humanities research engineering community can broaden to encompass academia, GLAM institutions and industry.

Exploration, optionality and much more

In summary, I’d like to make three points based on my experience at the Turing's Research Engineering Group. Firstly, that few jobs, especially after a PhD, allow one to widely explore interests and at the same time broaden professional options as much as research engineering. Secondly, that research engineering involves more than practicing a narrow set of technical skills; it fosters teamwork and values such as openness and collaboration. I went in to skill-up, and found so much more. Thirdly, that the digital humanities might benefit from creating its own approach to research engineering, as a way to incorporate some of these skills and practices, as well as developing much-needed research infrastructure.

Why, then, am I returning to academia? Put simply, this is where my inclinations have led me. While research engineering is definitely a great career choice for some, it can also be a fantastic stepping stone towards a career elsewhere, as I have argued here. This is the case for me.

Allow me, then, to praise research engineering as a profession, a community and a culture. And to the Turing's Research Engineering Group: thank you, it has been a blast.