Statistics and the law: Probabilistic modelling of forensic evidence

Introduction

Please note that this research is based on real casework, and the nature of the work is highly sensitive.

Forensic science plays an increasingly important role in contributing to national security resilience and the successful administration of the justice system. Major ongoing technological innovation has propelled forensic science development and sourced new types of forensic evidence. Such technological and scientific advances must be matched by well-developed tools for the analysis, evaluation and interpretation of evidence in order to successfully exploit innovations in the administration of justice.

Explaining the science

Forensic evidence

Multiple and different types of forensic evidence (e.g. DNA, fingerprints, mobile devices, eyewitness testimony) may be collected as part of a criminal (or civil) investigation. Forensic scientists are expected to evaluate evidence, providing often a numerical judgement about the value of the evidence in the relevant context. The evaluation of evidence often involves the comparison of a crime scene sample with a reference sample, and subsequently formulated as a likelihood ratio in which the interest is in the probability of the evidence under two competing propositions – a prosecution and defence or alternative proposition.

The widely accepted hierarchy of propositions to be addressed in casework involves the pairing, framing and development of at least two competing propositions to be done with careful consideration of the case circumstances and evidence. Forensic scientists are often challenged to justify their decision-making process and to answer a wide range of questions that involve considering activity level propositions, combination of evidence, use of quasi-experiments, expert judgement and non-random data sources.

Evaluating evidence

The evaluation of forensic evidence in cases often involves complex scenarios requiring the evaluation of more than one evidence type and/or addressing different proposition levels. In addressing activity level propositions or even a joint likelihood statement of observing a combination of evidence, it is important to recognise that data to calculate probabilities is often limited as case circumstances are often unique. Different scenarios, as presented by defence and prosecution, can introduce different sets of relevant information affecting the dependence relationship between variables. While there may be quasi-experiments and some relevant data available, numerical judgement at the activity level requires eliciting expert probabilities, managing uncertainty, and consideration of the decision-making process and causal notions.

Graphical models

Graphical models have gained prominence as decision-support systems which express the complex graphical and probabilistic relationships between measured variables, providing a graphical representation of the problem (or forensic case), calculating marginal and conditional probabilities of interest, and facilitating a meaningful and logical communication of evidential and statistical reasoning between researchers and practitioners.

Whilst graphical models provide a versatile, dynamic, modular and coherent framework, there are several statistical challenges including:

Qualitative and computational considerations in conditional independence relationships as evidence/sub-model combined.
Quantitative computation of combining likelihoods from different data structures.
Accounting for uncertainty from multiple and complex scenarios and data structures.
Determining how to incorporate a combination of elicited probabilities from experts/witnesses, quasi-experiments and observational data.
Accounting for time, sequential events and potential causal pathways.

Example case circumstances

Suppose an individual is accused of assaulting someone known to them. The accused person denies the assault and provides their account of the event and activities which led to their fingerprints being at the scene. What is the probability that the evidence arose under the victim’s, and thereby prosecution’s, account of the events? What is the probability that the evidence arose under the competing proposition(s) set forth by the defence based on the accused’s account? What other features of the evidence (besides those relevant to source identification), relevant data sources, contextual information, knowledge and expertise can inform these questions? Now suppose that the evidence relevant to the case was a digital device or multiple different types of forensic evidence. How does addressing individual source and activity level questions conflict or support competing propositions?

Project aims

This project aims to draw upon real cases and engage with forensic science, legal and policing community to develop a probabilistic framework for the evaluation of complex forensic evidence, that deals with the multiple statistical issues and complex data structures that can occur. There is particular interest in understanding activity level propositions, using graphical representations such as Bayesian networks, Wigmore charts and chain event graphs.

The specific goals are to:

Develop a coherent and systematic probabilistic framework for the interpretation of complex cases, that accounts for multiple types of evidence, addresses propositions at different levels, incorporates expert judgement and multiple sources of uncertainty, and considers and informs uncertainty in inferences from databases.
Develop statistical methods to address the computational complexities of combining different modelling substructures, to model evidence conflict, and to facilitate the modelling of the case circumstances.
Apply modern approaches to statistical causal reasoning to understand, model and assert the relevance of the evidence to the legal issues in a case.

Applications

This research will provide a probabilistic and coherent framework and tools for the evaluation and interpretation of evidence using graphical models. The probabilistic and graphical structure facilitates the laborious calculation of marginal and conditional probabilities of interest, and the logical statements and mechanism to communicate uncertainty and complex probabilistic statements between forensic scientists, lawyers, judges and jury. Such a framework and associated tool will inform the basis of training for forensic scientists and the legal community in probabilistic (and causal) reasoning in the interpretation of evidence, supporting the expected UK forensic regulator guidelines, and can be useful during a criminal (or civil) investigation, evaluation and for communication in courts.

Recent updates

December 2019

Seeking a full-time post-doctoral Research Associate (RA) to work on the project. The researcher will be part of a team of top academics in Bayesian statistics, decision theory and causal inference and be based at The Alan Turing Institute in London.

More information here and application form here.

October 2019

Research paper: Psychometric Analysis of Forensic Examiner Behaviour

July 2019

Research paper: Estimation of temperature-dependent growth profiles for the assessment of time of hatching in forensic entomology

April 2019

Delivery of two-day kick-off project workshop in April 2019 entitled 'Probabilistic reasoning and decision-making of forensic evidence – addressing activity level propositions

2018

Speakers at RSS meeting in December 2018
Contributed to House of Commons Science and Technology Report, September 2018

Activities and interests

Rapid growth of technological advances and its role in the development and application of forensic sciences as it intersects with the law and broader justice system possesses a multitude of challenging and interesting questions for legal, forensic science, policing, statistics, computer science researchers, practitioners and decision/policy-makers.

The Silent Witnesses research group are interested in a wider range of questions in which science, probability, data and algorithms meets justice. Some of the group's current and future activities include (but are not limited to):

Serving on the RSS Statistics and Law Section Committee, ICFIS scientific committee.
Advising on national and international policy and guidelines.
Serving as an expert witness.
Holding an annual workshop and quarterly roundtable meetings on special topics to engage wider research, practitioner, and policy engagement and opportunities for collaboration (beginning Spring 2020).
Investigating use of open data and models to deliver research results (Dundee), developing and delivering training exemplars with the Metropolitan Police and other forensic science providers, and engaging with broader justice, legal and security community to improve statistical reasoning and promote interdisciplinary research in the forensic sciences.