Introduction
Numerical simulations are increasingly used for engineering design, although they are not always accurate. To be accurate, simulations require faithful models of the underlying physics, which are usually expressed through partial differential equations (PDEs). Model parameters are obtained from experimental data but, for detailed simulations, this process is usually too expensive to be practical.
The aim of this project is to combine Probabilistic Programming with adjoint methods for PDEs in order to accelerate model selection and model tuning. This will widen the application of Probabilistic Programming to fields such as Fluid Dynamics and provide a framework to exploit the vast amounts of data produced by experiments and numerical simulations.
Explaining the science
Many traditional scientific methods codify physical principles, such as conservation of mass and momentum, into partial differential equations (PDEs), which are then solved numerically. Although these numerical solutions can be extremely detailed, they are not necessarily accurate. Their accuracy requires good models and accurate model parameters. This is why experimental testing remains a crucial component of the engineering design process. CFD simulations and experimental results are usually discarded once a design has been chosen, however, despite producing gigabytes of information that could be useful in the future. This project is partly motivated by a desire to use this data effectively and sustainably.
The aim of this project is to generate and popularise methods that exploit (i) the vast amounts of data generated from experiments and PDE solvers (ii) the high-quality physics-based priors that are expressed through PDEs, and (iii) recent developments in probabilistic programming. For this aim, a Bayesian framework is ideal: The PDEs is treated as a model in which some parameters are fixed while others are expressed as probability distributions. The prior distributions are asserted by the user and then, as data arrives, the posterior probability distributions of the parameters are updated deterministically.
The challenge with implementing PDEs within Probabilistic Programs is that function evaluations are usually prohibitively expensive. This means that traditional sampling methods such as Markov Chain Monte Carlo (MCMC) become intractable. PDEs present an opportunity, however, because the outputs of many PDEs vary smoothly with their parameters. For these PDEs, Gaussian prior distributions lead to nearly Gaussian posterior distributions, meaning that Laplace’s approximation becomes sufficiently accurate to be useful. This can be combined with adjoint methods such that tens of thousands of parameters of large PDE problems can feasibly be inferred cheaply, even when data is sparse. The Gaussian assumption can be tested a posteriori with MCMC if desired, albeit at considerable expense.
Project aims
In this project, we will extend Programmable Inference to large non-self-adjoint PDE problems, such as the Navier–Stokes equations, by implementing adjoint-accelerated Laplace’s method within the Probabilistic Program Turing.jl. We will also publish stand-alone examples in Matlab and Python in order to reach other research communities.
The technical aims are to: (i) re-structure our existing adjoint-accelerated inference codes within a generalized open-source programmable inference framework and implement this within Turing.jl; (ii) extend this framework to broad families of PDEs and eigenvalue problems such that it can be exploited beyond the field of Fluid Dynamics; (iii) develop three flagship applications of programmable inference and apply these to industrial problems relevant to the UK.
The strategic aims are to: (i) promote the development, adoption, and awareness of Programmable Inference and Probabilistic Programming; (ii) combine the UK’s existing strengths in Fluid Dynamics and data science, such that the UK becomes a world leader at the intersection of these large research areas; (iii) de-risk Probabilistic Programming for PDEs with proven test cases.
Applications
These activities will widen the application of Probabilistic Programming to the discipline of Fluid Dynamics. A 2021 report on the impact of Fluid Dynamics in the UK showed that Fluid Dynamics is central to many of the UK’s societal and industrial challenges, contributing £13.9 bn annually to the UK, and underpinning many high-value-add jobs outside the South-East. This discipline is central to aspects of the Turing 2.0 Grand Challenges of Health and Environment & Sustainability (e.g. next generation numerical weather prediction), and provides tools for Data-centric Engineering.
