In recent years we have seen outstanding progress in designing algorithms – sequences of instructions performed by a computer which enable it to process information.
In a high-performance computing environment, such as a data centre with hundreds or thousands of interconnected computers, algorithms allow huge data analysis tasks to be performed. For example, classifying millions of images of tissue samples to identify whether they contain anomalous features that should be examined by a doctor, and give an effective yes/no output.
While these high-performance systems operate well for some computing needs, they often run at less than half their full capacity for many data science tasks. This is caused by the way in which future computer systems are designed; they are typically developed so that they perform well for a number of model algorithms, and these do not include some of the most important data science algorithms today. At the same time, data science algorithms are designed so as to perform well on existing, rather than future, hardware. The absence of feedback between these two design tasks is impeding progress in high-performance, large-scale data analysis.
As data science continues to grow as an industry and research sector, data-driven algorithms such as those required by deep learning take up an increasing amount of valuable time and energy in data centres, provoking a need for computing companies to rethink how they manage the technical challenges caused by this emerging new science.
In this five-year project funded by Intel, scientists at the Turing are working to address this challenge through co-designing computer hardware and software. That is, hardware will be designed to suit the needs of data science algorithms, which will similarly be designed to suit the capabilities of the hardware. The research, once complete, promises to dramatically increase the speed and efficiency of data-driven computing tasks and will provide Intel with the tools to build the next generation of computer processors and high-performance systems.