As generative language models improve, they open up latest possibilities in fields as diverse as healthcare, law, education and science. But, as with all latest technology, it’s price considering how they could be misused. Against the backdrop of recurring online influence operations—covert or deceptive efforts to influence the opinions of a target market—the paper asks:

How might language models change influence operations, and what steps could be taken to mitigate this threat?

Our work brought together different backgrounds and expertise—researchers with grounding within the tactics, techniques, and procedures of online disinformation campaigns, in addition to machine learning experts within the generative artificial intelligence field—to base our evaluation on trends in each domains.

We imagine that it’s critical to research the specter of AI-enabled influence operations and description steps that could be taken before language models are used for influence operations at scale. We hope our research will inform policymakers which can be latest to the AI or disinformation fields, and spur in-depth research into potential mitigation strategies for AI developers, policymakers, and disinformation researchers.

How Could AI Affect Influence Operations?

When researchers evaluate influence operations, they consider the actors, behaviors, and content. The widespread availability of technology powered by language models has the potential to affect all three facets:

1. Actors: Language models could drive down the price of running influence operations, placing them nearby of recent actors and actor types. Likewise, propagandists-for-hire that automate production of text may gain latest competitive benefits.

2. Behavior: Influence operations with language models will develop into easier to scale, and tactics which can be currently expensive (e.g., generating personalized content) may develop into cheaper. Language models can also enable latest tactics to emerge—like real-time content generation in chatbots.

3. Content: Text creation tools powered by language models may generate more impactful or persuasive messaging in comparison with propagandists, especially those that lack requisite linguistic or cultural knowledge of their goal. They can also make influence operations less discoverable, since they repeatedly create latest content while not having to resort to copy-pasting and other noticeable time-saving behaviors.

Our bottom-line judgment is that language models will likely be useful for propagandists and can likely transform online influence operations. Even when probably the most advanced models are kept private or controlled through application programming interface (API) access, propagandists will likely gravitate towards open-source alternatives and nation states may spend money on the technology themselves.

Critical Unknowns

Many aspects impact whether, and the extent to which, language models will likely be utilized in influence operations. Our report dives into lots of these considerations. For instance:

• What latest capabilities for influence will emerge as a side effect of well-intentioned research or business investment? Which actors will make significant investments in language models?
• When will easy-to-use tools to generate text develop into publicly available? Will it’s simpler to engineer specific language models for influence operations, slightly than apply generic ones?
• Will norms develop that disincentivize actors who wage AI-enabled influence operations? How will actor intentions develop?

While we expect to see diffusion of the technology in addition to improvements within the usability, reliability, and efficiency of language models, many questions on the longer term remain unanswered. Because these are critical possibilities that may change how language models may impact influence operations, additional research to scale back uncertainty is very priceless.

A Framework for Mitigations

To chart a path forward, the report lays out key stages within the language model-to-influence operation pipeline. Each of those stages is a degree for potential mitigations.To successfully wage an influence operation leveraging a language model, propagandists would require that: (1) a model exists, (2) they will reliably access it, (3) they will disseminate content from the model, and (4) an end user is affected. Many possible mitigation strategies fall along these 4 steps, as shown below.

If a Mitigation Exists, is it Desirable?

Simply because a mitigation could reduce the specter of AI-enabled influence operations doesn’t mean that it must be put into place. Some mitigations carry their very own downside risks. Others might not be feasible. While we don’t explicitly endorse or rate mitigations, the paper provides a set of guiding questions for policymakers and others to contemplate:

• Technical Feasibility: Is the proposed mitigation technically feasible? Does it require significant changes to technical infrastructure?
• Social Feasibility: Is the mitigation feasible from a political, legal, and institutional perspective? Does it require costly coordination, are key actors incentivized to implement it, and is it actionable under existing law, regulation, and industry standards?
• Downside Risk: What are the potential negative impacts of the mitigation, and the way significant are they?
• Impact: How effective would a proposed mitigation be at reducing the threat?

We hope this framework will spur ideas for other mitigation strategies, and that the guiding questions will help relevant institutions begin to contemplate whether various mitigations are price pursuing.

This report is removed from the ultimate word on AI and the longer term of influence operations. Our aim is to define the current environment and to assist set an agenda for future research. We encourage anyone desirous about collaborating or discussing relevant projects to attach with us. For more, read the complete report here.