A previous blog post discussed how the NHS COVID-19 app is using the latest API from Google and Apple, combined with a novel application of a sequential Bayesian inference algorithm, to improve performance of risky encounter notifications – a world first. The NHS COVID-19 app has notification performance (linked to its ability to tell between separate high and medium risk categories) that is deemed to be “excellent” – see this unrelated reference for such terminology.
During the early stages of the pandemic, modelling and simulation based research suggested that “a contact-tracing app ... can achieve epidemic control if used by enough people”. We wanted to rigorously test and assess the strength of this assertion, using data from the NHS COVID-19 app.
Has the app played a role in reducing the spread of the virus?
We are pleased to share our latest research, which demonstrates that the app is having a positive effect on reducing the impact of the virus. We estimated that for every 1% increase in app users, the number of infections can be reduced by 0.8% (from modelling) or 2.3% (from statistical analysis). Modelling based on the observed notifications and secondary attack rate yielded 284,000 (224,000-344,000; 95% confidence interval) averted infections, and statistical comparison of matched neighbouring local authorities yielded 594,000 (317,000-914,000; 95% confidence interval) averted infections. Improvements in the app notification system introduced in late October 2020 translated to increased epidemiological effectiveness of the app, further supporting the evidence for a direct effect of the app.
The paper is submitted for formal peer review. However, prior to public dissemination, the paper has been shared with a number of independent scientists for review and feedback – we are grateful for their input to date.
We believe that this evidence supports the need for the continued promotion, adherence, and greater adoption of the NHS COVID-19 app (and other similar contact tracing apps around the world), to work alongside other non-pharmaceutical interventions, in order to help control the virus during these challenging winter months and beyond. In summary: use the app, it works.
What measures are in place to protect user privacy?
Before describing aspects of the app data, let’s briefly review the measures in place to protect user privacy. The app has been carefully designed and implemented in a way that uses digital infrastructure to promote individual and public health benefit, yet also preserve user privacy and anonymity. Details are contained within the published Data Protection Impact Assessment. However, here’s some of the key features that promote user privacy:
- Decentralised design: the decentralised design of the contact tracing architecture means that data cannot be used to reconstruct observations of social interactions. That is, as well as being designed to prevent user identification, the decentralised design also prevents anybody (even the users) knowing who had contact with who.
- Technical measures: in order to reduce privacy risk further, the app has implemented traffic-based obfuscation (so a well-resourced ‘adversary’ cannot infer anything from network traffic patterns) to mitigate more esoteric attacks.
- Minimal data collection: the app only collects data deemed absolutely necessary (as reviewed by several independent organisations) to help understand the spread of the virus and optimise the app in its role as a non-pharmaceutical intervention.
- No identifiers: data sent to the NHS secure analytics environment does not contain any form of identifier, unique or otherwise. Device IP address information is removed immediately too. Data cannot be correlated over time. From a user perspective, this means that data cannot be linked to an individual’s specific device.
- No tracking: the app does not track location, or in any way interact with the operating system’s GPS functionality. The user specifies the postal district (usually first three characters of postcode) and can update this if they wish. Information about which locations a user checked into using QR code functionality remains on the user’s device.
The app team has a good working relationship with the Information Commissioner’s Office (ICO). The National Cyber Security Centre has ensured that privacy and security is a priority. As explained in the Data Protection Impact Assessment, the app allows some limited data to be collected – which are aggregated at the postal district level to protect privacy – to enable the app team to assess whether it is working as expected, to gain insights into the virus, and to establish whether the app is impacting the spread of the virus. In the remainder of this blog post, we’ll provide details of some of these early insights, and our ongoing research plans.
What is the app telling us?
Let’s consider the user journey. It all starts with a download. At the time of writing, the app has been downloaded an incredible 21.63 million times (unique, non-repeat downloads) since launch. This represents 56% of the eligible population, aged 16+ with a smartphone, based on population data from the Office of National Statistics and data from OFCOM for smartphone usage.
We believe that most users will only need to interact with the app to check into a venue. This feature allows health officials to send venue alerts and advice to users, and for users to keep a private and secure digital log of the places that they have visited, should they ever need to report this information to contact tracers. Venue and timing information stays on the device – user information is in their hands only. Anonymised check-in count data are collected, which can act as an indicator of region-based social mixing. To late January 2021 there have been over 103 million user check-ins. There are definite patterns in these data: outside of national lockdowns, Saturday was the day of most check-ins, while Monday was the lowest. During December, check-ins in higher tier areas were lower than in lower tier areas, as one might expect. There was a significant drop in check-in count data during Wales’ firebreak and England’s second national lockdown, that is, November to early December – as expected. Check-in data also acts as a (loose) indicator of social mobility. The adjacent map shows the difference in the average number of venue check-ins per user between late November and early December (i.e. the difference between December and November check-in data). The impact of local and national restrictions can be clearly seen; a relaxation of restrictions in tier one and tier two areas has resulted in an increase in check-ins. The difference between tier one (e.g. Cornwall) and tier two areas is also clear. Wales and England’s tier three areas saw little change in check-in activity during this period.
Users will also interact with the app to enter symptoms – which can direct them to book a test via the website. Since launch, to the time of writing, there have been over 1.4 million reported COVID-19 linked symptoms using the app. Due to the privacy protection in place, it’s not possible to know how many of these users went on to book at test. During the same period, over 3.1 million app users used the app to record / receive a test result (not all tests are booked via the app, but users are still using it to record test results, which is reassuring). Of the 3.1 million that interacted with the app with their test results, approximately 26% were positive tests. This is extremely encouraging: the app is being actively used by users, and users appear to want to enable the app’s trace functionality in order to protect others.
The app’s trace functionality is where it can excel. It can notify 100% of users (with an internet connection) that have been in contact with a user that has tested positive (the index case) within a matter of hours of the test result being sent to that user (best case 15 minutes). The app has complete recall. It is able to process extremely large numbers of cases too, making it an ideal companion to manual tracing and expanding the reach of tracing efforts beyond that of traditional processes alone. The app will only notify users that are very likely at risk, in line with national guidance. If a user has not been near an index case, then the app will not notify them; in that sense it is 100% accurate. However, when a user has been near an index case, the app assesses the riskiness of the contact event, and sends notifications (in a privacy preserving manner) to users that are at risk. In total, using anonymised data from the app, estimates indicate that the app has issued over 1.7 million contact tracing notifications asking users to isolate, since launch. As one might expect, the weekly magnitude of this number follows the flow of England / Wales statistics. For example, during 24 – 30 December 2020, the app sent over 263,000 (England) and 8,600 (Wales) contact tracing notifications to isolate. Every one of these users is at risk, so this is an encouraging number from the perspective of app functionality.
Exposure Notifications Per Index Case (ENPIC) – a leading indicator
The decentralised design of the app means that it is not possible to directly measure the number of notifications that each individual index case generates. However, given anonymised counts of users who test positive and counts of the number of people asked to isolate due to risky contact, linked (via user supplied information) to local authorities, it is possible to estimate the average number of exposure notifications per index case. The intuition goes: a high ENPIC indicates a high number of risky interactions having taken place within a given geographical area (local health authority region). The time at which this value is observed is (usually) prior to the onset of symptoms for risky contacts. So, the ENPIC measure is an indicator of virus incidence; it can help identify areas where infection rates are more likely to grow or decline in the near future.
We estimate that the app has sent an average of 3.2 notifications to isolate per index case, or 4.4 notifications per index case who consented to be contact traced. Preliminary statistical analysis has shown that areas with a higher ratio of exposure notifications per index case experience a greater increase in the rate of infections than other parts of the country. Conversely, areas with the lowest rates of notifications have seen case numbers fall more quickly. For example, in early January, the ENPIC data pointed towards the north west and south west of England as being at-risk areas. Importantly, this signal appears to come approximately 5-7 days ahead of a change in case rates from app testing data.
This suggests that this inferred signal from the app can play a valuable role as a leading indicator for identifying areas at greater risk of a growth in cases and/or where restrictions on social and economic activity are successfully helping bring infections under control. We’re performing research into the app’s ability to provide additional information useful for estimating a local reproduction number. This could provide significant benefit to the local and national health policy teams.
Data insights summary
As with any statistical analysis, it is necessary to undertake a period of exploratory analysis, correct and model any data quality issues that arise, before producing definitive conclusions. The narrative above offers a number of insights into our exploratory analysis, to date. Researchers from The Alan Turing Institute in collaboration with Oxford University are performing additional analysis and seek to publish the research and findings in an open and reproducible manner. To summarise the data exploration: the empirical evidence suggests that the app is providing valuable public health insight (whilst protecting user privacy), and is working as expected, helping to keep people safe.
What research is next?
Data from the app may also have an important role to play in wider policy development and operational planning. For example, we can use the space-time statistical analysis to help to characterise “what is a safe distance?”, given unique distance-related data that the app is able to generate. This could also help to reshape policy, as necessary, helping inform future guidance relating to social distancing in specific contexts. This is an ongoing research exploration.
Researchers from The Alan Turing Institute in collaboration with Oxford University are performing a detailed space-time statistical analysis (based around latent Gaussian Markov random fields) of the data that are collected, which will help us to model and understand virus-related phenomena across space-time.
We have just finished producing preliminary research into the potential for a mobile device to infer whether an encounter takes place indoors or outdoors. This information could help to inform the risk calculation, and potentially wider public policy.
Acknowledgements
We are extremely grateful to the wider NHS COVID-19 app team who work tirelessly to ensure that digital contact tracing helps to control the virus (alongside all other non-pharmaceutical interventions). We acknowledge the many fruitful conversations with collaborators at Google and Apple. We also acknowledge the contributions by Rob Parker, Matthew Ayres, Dr Marcos Charalambides, Daphne Tsallis, Dr Chris Wymant, Dr Michelle Kendall, Dr Luca Ferretti and Dr Robert Hinch.
Additional information
See the project page for additional information about the Turing’s support to the NHS COVID-19 app team.
Cover photo by Keira Burton from Pexels.
