Promoting Online Safety by Simulating Unsafe Conversations with LLMs

Generative AI, including large language models (LLMs) have the potential -- and already are being used -- to increase the speed, scale, and types of unsafe conversations online. LLMs lower the barrier for entry for bad actors to create unsafe conversations in particular because of their ability to generate persuasive and human-like text. In our current work, we explore ways to promote online safety by teaching people about unsafe conversations that can occur online with and without LLMs. We build on prior work that shows that LLMs can successfully simulate scam conversations. We also leverage research in the learning sciences that shows that providing feedback on one's hypothetical actions can promote learning. In particular, we focus on simulating scam conversations using LLMs. Our work incorporates two LLMs that converse with each other to simulate realistic, unsafe conversations that people may encounter online between a scammer LLM and a target LLM but users of our system are asked provide feedback to the target LLM.

The Challenges of Studying Misinformation on Video-Sharing Platforms During Crises and Mass-Convergence Events

Mis- and disinformation can spread rapidly on video-sharing platforms (VSPs). Despite the growing use of VSPs, there has not been a proportional increase in our ability to understand this medium and the messages conveyed through it. In this work, we draw on our prior experiences to outline three core challenges faced in studying VSPs in high-stakes and fast-paced settings: (1) navigating the unique affordances of VSPs, (2) understanding VSP content and determining its authenticity, and (3) novel user behaviors on VSPs for spreading misinformation. By highlighting these challenges, we hope that researchers can reflect on how to adapt existing research methods and tools to these new contexts, or develop entirely new ones.

Sedition Hunters: A Quantitative Study of the Crowdsourced Investigation into the 2021 U.S. Capitol Attack

Social media platforms have enabled extremists to organize violent events, such as the 2021 U.S. Capitol Attack. Simultaneously, these platforms enable professional investigators and amateur sleuths to collaboratively collect and identify imagery of suspects with the goal of holding them accountable for their actions. Through a case study of Sedition Hunters, a Twitter community whose goal is to identify individuals who participated in the 2021 U.S. Capitol Attack, we explore what are the main topics or targets of the community, who participates in the community, and how. Using topic modeling, we find that information sharing is the main focus of the community. We also note an increase in awareness of privacy concerns. Furthermore, using social network analysis, we show how some participants played important roles in the community. Finally, we discuss implications for the content and structure of online crowdsourced investigations.