AI for propaganda

In Brief

With the onset of large language models, the mass production of text has become considerably cheaper and faster, requiring less human involvement. With the addition of automatic image generators such as Dall-E, the same is now possible for pictures. Previous work has found that GPT-3, the antecedent to ChatGPT is capable of creating text equally persuasive as content from existing covert propaganda [1]. This causes a rising concern regarding the ease at which a large number of influential texts can be circulated using online platforms. To combat mass-propaganda, it is important that online and offline platforms have effective controls to identify AI-generated materials in their publications.

More in Details

Propaganda as a tool for political persuasion is not novel and has been used to spread political information for centuries. However, with the onset of the AI revolution, the generation and publication of propaganda has been greatly eased. As a consequence, there is a rise in concerns about the impact and negative ramifications of AI-generated political messaging. The following first discusses the use of Large Language Models in the generation of propaganda and their perception by the public. We then discuss the use of recommender systems to create a political echo chamber. Finally we outline the section of the EU AI act which may be applicable to AI-generated propaganda.

Large Language Models

The high quality of text produced by large language models such as ChatGPT has caused rising concerns about an increasing amount of AI-generated content circulating online. While careful analysis and machine learning models may allow for the differentiation of AI and Human generated text, this may not be possible for end-users \cite{liao2023differentiating, herbold2023large}. Additionally, some preliminary research has found that given the correct prompts and some baseline human scanning, large language models are capable of producing text equally persuasive to human-generated covert propaganda campaigns [1]. Consequently it is vital that AI-generated texts and images be identifiable in both online and offline forums. While generated text may not be intentionally biased or aimed to be propaganda, it may still hold implicit biases, which could be influential all the same if not prefaced properly. As such, any AI which is designed to generate content that directly interacts with the end-user should clearly identify the content which is AI generated. Furthermore, providers of online and offline forums, which may contain AI-generated materials should set in-place validated mechanisms for evaluating published content for AI-generated materials and mark the content appropriately.

Recommender Systems

Recommender Systems are AI algorithms which aim to show users content that is consistent with their previous interactions. Such algorithms are commonplace on platforms such as Netflix and Instagram, where recommended content is based on the user’s interaction with previous posts. These algorithms can become instrumental for political propaganda in two ways. The first is through amplification. Here the algorithm reinforces extremist political tendencies by presenting content consistent with these ideals, leading to a metaphorical echo chamber. As a consequence, users’ views are reinforced, leading to increasing extremism and a lack of confrontation with opposing views. Alternatively, recommender systems may cause persuasion through repeatedly exposing the user to certain content. During this process, neutral users are increasingly recommended extremist content, despite no previous interaction with this material. As a result users may become persuaded by the content, leading to involuntary opinion change. The presentation of extremist content through recommender systems may result from system bias or purposeful manipulation. Research has found that some recommender systems appear to be more prone to these biases than others, indicating that training data and user base may impact the algorithm’s function [2].

Guidelines

The current AI act guidelines do not explicitly stipulate regulations relating to the AI-based generation of propaganda. However, we argue that this should be placed in the unacceptable risk level bracket as it may exploit vulnerable groups through targeted advertisement and automatically generated content. As such, the automatic generation and circulation of propaganda should be outlawed under EU legislation. However, the enforcement of such rules are questionable due to the intricacies of propaganda. As such, more research in the identification and reporting of AI generated and circulated propaganda is required.

Bibliography

1(1,2)

Josh A Goldstein, Jason Chao, Shelby Grossman, Alex Stamos, and Michael Tomz. How persuasive is ai-generated propaganda? PNAS nexus, 3(2):pgae034, 2024.

2

Joe Whittaker, Seán Looney, Alastair Reed, Fabio Votta, and others. Recommender systems and the amplification of extremist content. Internet Policy Review, 2021.

This entry was written by Nicola Rossberg and Andrea Visentin.