The clock is ticking to the US election, and AI-created disinformation is never far from the news. So far, we’ve had deepfake robocalls, false celebrity endorsements, uncensored chatbots and fake news from Russia-based websites. And while fears of the effects of AI on the 2024 UK general election were largely unfulfilled, time will tell just how much impact this ever-evolving technology is having across the Atlantic.
The use of technology to mislead voters is not a new phenomenon: past elections have been targeted by malicious actors looking to influence the opinions of the electorate through organised social media campaigns, using ‘bot’ accounts to spread disinformation. But ‘generative AI’ technologies such as large language models (LLMs) and chatbots offer new opportunities for creating believable, high-quality content tailored to specific audiences, all in a matter of seconds.
Whereas more traditional disinformation campaigns might copy and paste the same sentences on social media with the occasional keyword change, LLMs are capable of adding contextual nuance and details, or adopting different tones and personas, potentially making the content more convincing and difficult to detect. This is already happening: OpenAI (the creators of ChatGPT), for example, revealed that its models have been used in support of deceptive online activity around issues including politics in Europe and the US, and Russia’s invasion of Ukraine.
How readily do LLMs create disinformation?
To understand the risks of LLMs being used in disinformation campaigns, we carried out a two-part study, first looking at how readily LLMs comply with requests to generate misleading content.
We asked 13 LLMs released over the past five years to create two types of disinformation within the context of UK politics: 1) fictitious claims about politicians, which could sow distrust and affect voting intentions, and 2) false information about voting logistics in specific areas, which could confuse voters. (Note that none of this disinformation was published online – it was purely created for the purposes of our study.)
We built an evaluation dataset that contained over 2,000 LLM prompts for these two types of scenarios, altering variables such as MP name, UK town and political perspective (i.e. left- or right-wing). The prompts encompassed multiple aspects of a disinformation campaign, from writing fake news articles and social media profiles to social media posts and replies.
Most LLMs go through safety training processes to prevent them from complying with prompts that are deemed harmful, but we found that the majority of LLMs complied with our instructions (only 3 out of the 13 LLMs refused more than 5% of prompts). The few models that refused malicious prompts also refused more benign election-related prompts (i.e. they were more sensitive to election content in general), and were more likely to refuse to generate content from a right-wing perspective.
How human is AI-generated content?
Creating disinformation with LLMs is one thing, but in order for a campaign to be successful, the disinformation has to be convincing.
In the second part of our study, we evaluated the quality of the text generated by the 13 LLMs, presenting over 700 human participants with election disinformation that had been written either by humans or LLMs, and asking them to label it accordingly. Our aim was to understand the extent to which AI-generated content can pass as being written by a human.
We found that the majority of models (9 out of 13) produced content that people were unable to discern from human-written examples at least 50% of the time. Two of these models (Gemini 1.0 Pro and Llama 3 70B) were more frequently labelled as ‘human’ than actual human-written content.
Our findings suggest that LLMs are already capable of producing text that most people cannot identify as AI-generated. Disinformation created by LLMs is thus likely to appear authentic and potentially have more impact than more traditional campaigns.
We also found that, when presented with pieces of content written by humans or LLMs, the more people mislabelled the AI-generated content as human-written, the more they mislabelled human-written content as AI-generated. This suggests that, in addition to enabling disinformation, sophisticated AI models may start to degrade trust in genuine human content.
Another factor that could make LLMs attractive to disinformation creators is their low cost. In our paper, we estimate that a campaign that would cost around US$4,500 using traditional methods could be achieved using a commercial, state-of-the-art LLM for less than US$1. There are also open-source LLMs that can be run locally on a laptop, theoretically bringing the costs of generating content to zero.
What can we do about it?
There is no simple solution to the problem of AI-generated disinformation. The LLM prompts used to generate content for a disinformation operation may not appear implicitly harmful or malicious, so it isn’t reasonable to expect models themselves to be able to identify when a prompt is being used for nefarious purposes. Identifying patterns of malicious use is a more feasible task, and some AI developers have done just so, but this isn’t possible when considering open-source models run locally.
It’s important for AI developers to make their models as safe as realistically possible, but tackling disinformation will ultimately be a group effort. Social media platforms need to do more to identify and prevent malicious activity, while policy makers and regulators have a responsibility to develop legislation that minimises the negative impacts of generative AI. Education also has a major role to play, through initiatives that enable people to better distinguish genuine content from disinformation. Finally, we mustn’t lose sight of the fact that generative AI also offers benefits to democracy and society by, for example, helping the repressed to spread their messages without fear of persecution.
Read the paper:
Large language models can consistently generate high-quality content for election disinformation operations
Top image: freshidea
