Introduction
The United Nation’s Sustainable Development Goals (SDGs) aim to “to promote prosperity while protecting the planet”. These SDGs address the many global challenges faced by humanity, such as poverty, inequality, access to healthcare and education, climate change, environmental degradation, building resilient infrastructure, creating strong institutions, and many more. All over the world, countries are striving to meeting the targets laid out in the SDGs by 2030 – an ambitious agenda by any measure.
Whatever the country, sustainable development is a transformational process that hinges primarily on one input: the provision of targeted government funding and resources. Progress towards the 17 SDGs are monitored by the collection of data on 231 “development indicators” (DIs) – though not all DIs are collected by all countries. Prioritising which policy issues to pursue for maximum impact is an enormous challenge for governments. That’s because the range of development policy options is countless, often with unanticipated inefficiencies that waste resources. And, crucially, there are complex interdependencies between policies that should be taken into consideration.
Modelling these sorts of complex scenarios effectively is impossible using traditional economics and statistical techniques. But this is exactly the sort of ‘wicked’, long-running policy challenge that cutting-edge data science and artificial intelligence technology can make a huge impact on. The Alan Turing Institute’s Public Policy programme is committed to working with policy makers around the world on data-driven policy innovation to address problems like this. An integral part of the programme is a project led by ESRC-Turing Fellow Omar Guerrero, with his research partner, Professor Gonzalo Castañeda of the Center for Research and Teaching in Economics in Mexico. Together, they have developed a suite of analytical tools that can successfully model the impact of a variety of policy decisions on development indicators.
In collaboration with the United Nations Development Programme (UNDP), this technology, called Policy Priority Inference, is already being adopted by state governments in Latin America to support the effective prioritisation of their public policies to optimise sustainable development. “The results of this project show the potential the Policy Priority Inference model has for providing governments with concrete information on how to increase the effectiveness of public spending and accelerate the achievement of development goals,” says Annabelle Sulmont, Public Policy Project Coordinator for the UNDP office in Mexico. “The model also provides a common language that enables its implementation in other parts of the world, and facilitates sharing and comparing results across regions and countries.”
Clearly, this is just the beginning. The new approach has the potential to supercharge the effectiveness of government-backed sustainable development all over the world, to the benefit of billions of people – and the planet itself.
"The results of this project show the potential the Policy Priority Inference model has for providing governments with concrete information on how to increase the effectiveness of public spending and accelerate the achievement of development goals.”
Annabelle Sulmont, Public Policy Project Coordinator for the UNDP office in Mexico
How did it start?
Decades ago, economists tended to focus on GDP as a measure of development, but this is a blunt, unidimensional measure. Today, the SDGs and their hundreds of associated DIs bring serious multidimensionality to the picture of development, and enormous complexity in how these many DIs are interconnected. For example, investing in industrialisation tends also to produce negative outcomes for the environment, while investing in public transport might also boost education outcomes because more children become able to access school.
These positive and negative ‘spillover’ effects are known in development circles as ‘interlinkages’. Imagine hundreds of development indicators, many linked to a lesser or greater degree, positively or negatively, and you get a sense of the scale of the challenge of development planning. “We cannot properly understand development if we don't understand the trade-off of investing in certain areas,” says Guerrero, who is also a Senior Research Fellow in the Economics Department of UCL.
“We cannot properly understand development if we don't understand the trade-off of investing in certain areas."
ESRC-Turing Fellow Omar Guerrero
The UNDP coordinates the UN’s activities linked to the SDGs. The UNDP’s Latin America bureau was scouting for new analytical methodologies to tackle with the multidimensional nature of the SDGs when they came across Guerrero and Castañeda’s earlier work, in which they modelled socioeconomic indicators and their interactions. “They found it highly relevant because Latin America is leading in the generation of certain types of data on development indicators, but they had no tools to deal with its multidimensionality and complexity,” says Guerrero. After a series of seminars by the researchers at the UN in New York, the UNDP asked the pair if they could refine their method and apply it to the SDGs. This was music to the ears of the researchers, who got into development economics in the first place, says Guerrero, “because it’s one of the fields in economics that have the most impact”.
The project officially began in March 2019, in collaboration with the UNDP office in Mexico, and the country’s National Laboratory for Public Policy at the Center for Research and Teaching in Economics (CIDE).
What happened?
Policy Priority Inference builds on a behavioural computational model, taking into account the learning process of public officials, coordination problems, incomplete information, and imperfect governmental monitoring mechanisms. The approach is a unique mix of economic theory, behavioural economics, network science and agent-based modelling. The data that feeds the model for a specific country (or a sub-national unit, such as a state) includes measures of the country’s DIs and how they have moved over the years, specified government policy goals in relation to DIs, the quality of government monitoring of expenditure, and the quality of the country’s rule of law.
From these data alone – and, crucially, with no specific information on government expenditure, which is rarely made available – the model can infer the transformative resources a country has historically allocated to transform its SDGs, and assess the importance of SDG interlinkages between DIs. Importantly, it can also reveal where previously hidden inefficiencies lie.
How does it work? The researchers modelled the socioeconomic mechanisms of the policy-making process using agent-computing simulation. They created a simulator featuring an agent called “Government”, which makes decisions about how to allocate public expenditure, and agents called “Bureaucrats”, each of which is essentially a policy-maker linked to a single DI. If a Bureaucrat is allocated some resource, they will use a portion of it to improve their DI, with the rest lost to some degree of inefficiency (in reality, inefficiencies range from simple corruption to poor quality policies and inefficient government departments).
How much resource a Bureaucrat puts towards moving their DI depends on that agent’s experience: if becoming inefficient pays off, they'll keep doing it. During the process, Government monitors the Bureaucrats, occasionally punishing inefficient ones, who may then improve their behaviour. In the model, a Bureaucrat’s chances of getting caught is linked to the quality of a government’s real-world monitoring of expenditure, and the extent to which they are punished is reflected in the strength of that country’s rule of law.
When the historical movements of a country’s DIs are reproduced through the internal workings of the model, the researchers have a powerful proxy for the real-world relationships between government activity, the movement of DIs, and the effects of the interlinkages between DIs, all of which are unique to that country. “Once we can match outcomes, we can discern something that's going on in reality. But the fact that the method is matching the dynamics of real-world development indicators is just one of multiple ways that we validate our results,” Guerrero notes. This proxy can then be used to project which policy areas should be prioritised in future to best achieve the government’s specified development goals, including predictions of likely timescales.
What’s more, in combination with techniques from evolutionary computation, the model can identify DIs that are linked to large positive spillover effects. These DIs are dubbed “accelerators”. Targeting government resources at such development accelerators fosters not only more rapid results, but also more generalised development. Guerrero has been in talks with the UK’s Department for International Development, which is particularly interested in Policy Priority Inference’s ability to identify development accelerators.
Impact in Mexico
Meanwhile, Guerrero and Castañeda have produced several reports with the UNDP, covering case studies from Mexico and Uruguay, and are now starting a project with Colombia, at the national and city levels. For the project with Mexico, the team first ran workshops with stakeholders from the office of the president, the finance ministry, the national institute of statistics, representatives of six of Mexico’s 32 states, and NGOs.
“The workshops were a tremendous success,” says Guerrero. “When we presented preliminary results for the federal case, they expressed that the model has great potential to support them in their budgeting and planning processes. The Mexican state governments provided us with valuable data to help test the model. They are keen on adopting Policy Priority Inference as part of their toolkit for when they plan budgets, and also when they prepare their own, state-level development plans. It helps them decide whether their goals are realistic or not.”
“The contributions of the researchers and the UNDP office in Mexico were mutually complementary,” Sulmont adds. “The UNDP’s experience in public planning and the 2030 Agenda, and its contact with the federal and state governments, assisted both academics in the adaptation of their complex model to the reality of Mexican public planning processes.”
“Participating in the project is extremely valuable. An exercise of this kind obliges us to be in the state of the art of monitoring and implementation of the 2030 Agenda.”
Mauricio Francisco Coronado García, Evaluation Director, Nuevo León’s Executive Office of the Governor, Mexico
So far, several state governments in Mexico have already expressed interest in adopting Policy Priority Inference. “Participating in the project is extremely valuable,” says Mauricio Francisco Coronado García, Evaluation Director in the state of Nuevo León’s Executive Office of the Governor. “An exercise of this kind obliges us to be in the state of the art of monitoring and implementation of the 2030 Agenda.”
He explains how his government wants to put Policy Priority Inference to use. “The current state administration is in the fourth of six years of government and about to carry out a 20/21 Agenda, a type of an administration-closure planning tool. In this, we could include inputs from Policy Priority Inference to: assess whether the goals set at the beginning of the administration are attainable according to the historical distribution of the transformative budget; make adjustments to the goals considering the above, and; use it to make a budget distribution proposal to the transition team of the next administration, according to the development profile they propose.”
What does the future hold?
So far, the UNDP’s Latin America bureau has been investing in the tools and the application of Policy Priority Inference in the continent has quickly spread beyond Mexico. If and when the technology becomes widespread in Latin America, the intention is to bring it to other global regions.
“It's very energising,” says Guerrero. “There is nothing else like this, and I'm always excited to see what the science will bring up in the next iteration.” One refinement the researchers are working on has become possible because some national governments are starting to report their expenditure data, says Guerrero. He recently received a three-year grant from the UK’s Economic and Social Research Council to take the Policy Priority Inference framework and merge it with this emerging fiscal data. Once the model can generate outcomes that not only match a country’s historical DI movements, but also tie into its real-world expenditure, confidence in the accuracy of its projections grows even more.
Guerrero is working on this with the UNDP and the Global Initiative for Fiscal Transparency (GIFT, formerly part of The World Bank). “Since my first chat with Omar about the project, I immediately saw a lot of potential in mutual collaboration,” says Lorena Rivero del Paso, Manager for Technical Cooperation and Collaboration at GIFT. “GIFT supports the data standardization of public finances and that is exactly what the Policy Priority Inference project needs. It’s a perfect complement. The project will be a good use case for the publication of spending open data, where structuring data can lead to better analysis and recommendations, which we would expect will turn into better policies for sustainable development.”
“Government expenditure data will take this technology to a whole new level,” says Guerrero. Policy Priority Inference is not only about government, Guerrero stresses, but also about accountability. “We want to bring these tools to NGOs too, because this is useful for them to assess the actions of governments. NGOs can check if governments are prioritising the right policies.”
Later in 2020, Guerrero plans to create a website with open-source tools, linked to the UN’s database of development indicators, so that interested researchers and analysts can explore their own policy scenarios for their country of interest. There will be tutorials published later this year on how to use the Policy Priority Inference tools in the Python programming language.
The Turing’s mission is to pioneer data science and artificial intelligence in order to change the world for the better. The Institute’s Public Policy programme contributes to this by developing research, tools and techniques that help governments design better policies and more effective public services. This work, with its potential to benefit everyone touched by the UN’s Sustainable Development Goals, is exactly the sort of work the Institute is proud to champion.
PDF Summary
Related people
