Can justice be blind when it comes to machine learning?

Researchers present findings at ICML 2018

Wednesday 11 Jul 2018

Concerns are rising that machine learning systems that make or support important decisions and judgments affecting individuals—e.g. by assessing the likelihood of criminals to reoffend, or by deciding which resumes should get filtered out during a job selection process—unfairly discriminate against certain groups. For instance there is evidence that COMPAS, an algorithmic tool used in some US jurisdictions to predict reoffending, is ‘biased against blacks’, or may be no more accurate or fair than predictions made by people with little or no criminal justice expertise.

A new paper, born from an idea at the Turing, and which will be presented this Friday 13 July at the International Conference on Machine Learning (ICML), introduces a new take on machine learning fairness. Authors from the Turing, Cambridge, Warwick, UCL and Max Planck Institutes for Intelligent Systems and Software Systems propose new methods to help regulators provide better oversight, practitioners to develop fair and privacy-preserving data analyses, and users to retain control over data they consider highly sensitive. By encrypting sensitive attributes, an outcome-based fair model may be learned, checked, or have its outputs verified and held to account, without users revealing their sensitive attributes in the clear.

Earlier work explored how to train machine learning models which are fair for particular subgroups of the population, such as gender or race (called ‘sensitive attributes’). To do so, these methods aim to avoid certain criteria such as ‘disparate impact’ and ‘disparate treatment’. To avoid disparate treatment, sensitive attributes should not be considered. On the other hand, in order to avoid disparate impact, sensitive attributes must be examined—e.g., in order to learn a fair model, or to check if a given model is fair. The authors introduce methods from secure multi-party computation which avoid both.

Lead author Niki Kilbertus says:

‘The field of fair learning has suffered from a dilemma: to enforce fairness, sensitive attributes must be examined; yet in many situations, users may feel uncomfortable in revealing these data, or companies may be legally restricted in collecting and utilising them, especially with the advent of GDPR. In this work we present a way to address this dilemma: by extending methods from secure multi-party computation, we enable a fair model to be learned or verified without users revealing their sensitive attributes.’

Turing Research Fellow Adrià Gascón says:

‘Recent developments in cryptography, and more concretely secure multi-party computation, are opening exciting avenues for regulators, as they can now audit and oversee sensitive information in a way that was never before possible. And while issues of social fairness are complex, we have put forward this approach as one tool which may be useful to mitigate certain concerns around machine learning and society.’

The authors work in a multidisciplinary fashion to connect concerns in privacy, security, algorithmic fairness, and accountability and this research is part of a larger body of work being explored at the Turing to address the challenge to make algorithmic systems fair, transparent and ethical. It is our aspiration to design and deliver ethical approaches to data science and AI by bringing together cutting edge technical skills with expertise in ethics, law, social science and policy. These themes relate to our data ethics group, our fairness, transparency and privacy group and connect to our efforts to manage security in an insecure world.

Notes to Editors:

1. Blind justice: Fairness with encrypted sensitive attributes will be published at Proceedings of the 35th International Conference on Machine Learning (ICML), Stockholm, Sweden and the project idea was born at The Alan Turing Institute.

2. The authors are Niki Kilbertus (Max Planck Institute for Intelligent Systems and University of Cambridge), Adrià Gascón (The Alan Turing Institute and University of Warwick), Matt Kusner (The Alan Turing Institute and University of Warwick), Michael Veale (University College London), Krishna P. Gummadi (Max Planck Institute for Software Systems), Adrian Weller (The Alan Turing Institute and University of Cambridge).

3. The authors’ extension of recent theoretical developments in secure multi-party computing to admit linear constraints may be of independent interest. They demonstrate the potential for real-world efficacy of their methods, and make their code publicly available at 

4. A selection of other papers accepted at ICML by Turing authors include: