Introduction
AI models that can generate photorealistic images, imitate the style of contemporary rap artists, or fluently answer queries about seemingly any topic have been the subject of countless recent headlines. This has led to an explosion of interest in foundation models -- the models that underpin these new tools. As many individuals and organisations question what these technologies might mean for their situations and sectors, regulators have also started to grapple with what this means for current and future regulation, such as the proposed EU AI Act.
To discuss the opportunities, challenges and potential solutions for governing these models, Turing’s Safe and Ethical AI hosted How To Regulate Foundation Models - Can we do better than the EU AI Act? This event featured a seminar by Lilan Edwards (Turing/Newcastle), and panel discussion together with Arnav Joshi (Clifford Chance), Carlos Muñoz Ferrandis (BigScience) and William Isaac (DeepMind), hosted by Adrian Weller (Turing/Cambridge). In this piece, we touch on just some of the issues and potential solutions discussed at the event.
Features of foundation models
Foundation models form the basis of many generative models -- models that can create outputs such as text, image, code or music. Recent high profile examples include OpenAI’s ChatGPT and Stability AI’s Stable Diffusion. Large foundation models are trained on vast amounts of data, often scraped from the internet, and require huge amounts of compute to train. The term ‘foundation models’ reflects how such models can form the basis of a wide range of applications: a language based foundation model can underpin systems used for search, customer support tools, or employment decisions. These fundamental features of foundation models present exciting opportunities, but also difficult challenges for regulators: the training datasets are too large to be manually reviewed, and raise questions about consent, copyright, protection for personal data, and liability. The computational requirements are only accessible to a small number of tech firms, entrenching the dominance of big tech and raising questions about competition, the future of open source models, and environmental harms. The general purpose nature raises questions about risks and liability -- where does responsibility lie between those who develop foundation models, and those who deploy downstream applications? Developers have knowledge and control over the dataset, and the largest financial gains, but can they be held responsible for unforeseeable uses and associated risks? Deployers have greater knowledge and control over their specific context and application, but it may be impossible for them to fix or even audit issues without access to upstream code and data.
Regulating foundation models
We have seen increasing recognition of the potential risks of such models. Stochastic Parrots by Emily Bender et al. articulates how such models can propagate and exacerbate bias, discrimination, misrepresentation, harmful stereotypes, and hate speech -- risks that the AI Act attempts to address. We have witnessed increasing recognition of other issues, such as (i) deepfakes, copyright, and the rights of artists and other contributors whose works are used for training; (ii) fake news and ‘hallucinations’, where models output false information fluently and convincingly; and (iii) models built on the assumption that data that is publicly available on the internet is fair game for developers, regardless of whether it contains personal or sensitive information. These developments raise challenges for the EU AI Act, a risk based approach that is characterised by the intended use of systems. General purpose models do not fit neatly into this paradigm, and appropriate requirements (for example those around data and data governance) will be challenging to define, and perhaps measure. Systems using foundation models represent a new value-chain model compared to specific use cases such as facial recognition, which have previously been the focus of discussions around risks from AI.
Where we go from here
Regulators are working hard to make the AI Act applicable to foundation models, though many questions still remain about appropriate requirements, and even how to define foundation models. Alternative regulatory approaches are being developed in different jurisdictions, such as the US (a more self-regulation centred approach), the UK (an approach based on existing regulatory capacity in different sectors, without new law) and China (a more ‘vertical’ technology-specific approach). As an alternative to the EU approach of developing new regulation, there are a number of existing legal mechanisms that could be applied or extended to foundation models, such as existing discrimination and equality law, the digital services act, privacy and data protection law, competition law, and regulation concerning product liability and copyright.
Alongside legislation, we need to develop complementary mechanisms to help understand and mitigate the potential risks of foundation models without unnecessarily stifling beneficial innovation. This includes developing and standardising effective evaluation strategies to analyse harms and articulate these to the public. These are needed to evaluate the models themselves, as well as to evaluate risks and harms that emerge from applications during deployment. Other proposed solutions include: licences that promote open source data and models while respecting responsible use; tools to allow content creators to opt-out of contributing their content to training data; and tools that allow end users to check the provenance and licences associated with generated outputs.
While many uncertainties and open questions remain, one thing is clear: regulators will continue to face the ‘pacing problem’, in which technological innovation threatens to outpace regulation and other necessary oversight mechanisms. In response, we need a collaborative effort, involving broad societal perspectives, and experts in technical issues, ethics, society, and policy, to provide what foresight we can, set some ground rules as early as possible, and be ready to iterate as we learn more about the technology, its applications, and its real world impacts.