The (mis)informed citizen

Using computational approaches to evaluate the quality of online news and studying its impacts

Introduction

Though concerns about the use of misinformation are certainly not new, the reach, speed and volume of misinformation in the digital era have generated a heightened sense of urgency among policymakers, scholars and the public alike. At the same time, relatively little is known about how such information is processed by people online – where much of it is encountered incidentally by citizens who are otherwise inattentive to public affairs – nor about its lasting effects on those who encounter it. This project therefore aims to develop a better understanding of the extent and impact of both quality and false, or misleading, information in online political news.

Explaining the science

The project is interdisciplinary in nature, combining theory and methodology from the social sciences, humanities and computer science. It builds a strong conceptual foundation on normative and empirical theories of news quality that are found in journalism and media studies, as well as political communication research.

It draws on a variety of theories from social and cognitive psychology, plus communication, political and information sciences in its exploration of the effects of exposure to online news reporting. The project’s methodological approaches brings together insights from communication science and natural language processing.

Project aims

The project’s aims are threefold. In a first step, it seeks to build automatic classifiers for signals of news quality. Recognising that the quality of reporting can vary from one story to the next – even within the same media outlet – this project will assess the quality of reporting at the article level.

Taking journalistic standards rooted in media theory as a starting point, these standards will be used to build classifiers for different elements of a story that may be more or less problematic. For example, classifiers for (a) the number and types of sources that are used to support claims and (b) the use of hyperbole in an article, among others. These indicators will then be combined to arrive at a holistic assessment of the quality of each news item.

The second phase of the project will assess whether and how strongly these elements correlate with the presence of false or misleading claims in a story. The goal in this second stage of research is to provide probabilistic assessments of how likely a story is to contain misinformation.

The project’s final phase will gather web trace (e.g., URLs visited) and survey questionnaire data from representative samples of members of the public, and deploy the developed news quality classifiers to assess both (a) the extent of people’s exposure to online news of varying quality and veracity and (b) the effects of such exposure on survey respondents’ political knowledge, opinions, and behaviours.

The project is initially examining news quality and misinformation in the United States, but additional case studies are planned.

Applications

The project is relevant to, and has broad applications for, media institutions and journalists, fact-checking organisations, social media and other digital platforms. It's also relevant to policymakers who hope to better understand and assess the online information landscape, as well as the real-world impacts that misinformation and other forms of low-quality information have on members of the public. The project will inform policy debates about the spread and effects of false news, as well as journalistic discussions about the shape and impact of quality reporting.

Organisers

Researchers and collaborators

Contact info

[email protected]