Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks

Large Language Models (LLMs) are swiftly advancing in architecture and capability, and as they integrate more deeply into complex systems, the urgency to scrutinize their security properties grows. This paper surveys research in the emerging interdisciplinary field of adversarial attacks on LLMs, a subfield of trustworthy ML, combining the perspectives of Natural Language Processing and Security. Prior work has shown that even safety-aligned LLMs (via instruction tuning and reinforcement learning through human feedback) can be susceptible to adversarial attacks, which exploit weaknesses and mislead AI systems, as evidenced by the prevalence of `jailbreak' attacks on models like ChatGPT and Bard. In this survey, we first provide an overview of large language models, describe their safety alignment, and categorize existing research based on various learning structures: textual-only attacks, multi-modal attacks, and additional attack methods specifically targeting complex systems, such as federated learning or multi-agent systems. We also offer comprehensive remarks on works that focus on the fundamental sources of vulnerabilities and potential defenses. To make this field more accessible to newcomers, we present a systematic review of existing works, a structured typology of adversarial attack concepts, and additional resources, including slides for presentations on related topics at the 62nd Annual Meeting of the Association for Computational Linguistics (ACL'24).

DeepMem: ML Models as storage channels and their (mis-)applications

Machine learning (ML) models are overparameterized to support generality and avoid overfitting. Prior works have shown that these additional parameters can be used for both malicious (e.g., hiding a model covertly within a trained model) and beneficial purposes (e.g., watermarking a model). In this paper, we propose a novel information theoretic perspective of the problem; we consider the ML model as a storage channel with a capacity that increases with overparameterization. Specifically, we consider a sender that embeds arbitrary information in the model at training time, which can be extracted by a receiver with a black-box access to the deployed model. We derive an upper bound on the capacity of the channel based on the number of available parameters. We then explore black-box write and read primitives that allow the attacker to: (i) store data in an optimized way within the model by augmenting the training data at the transmitter side, and (ii) to read it by querying the model after it is deployed. We also analyze the detectability of the writing primitive and consider a new version of the problem which takes information storage covertness into account. Specifically, to obtain storage covertness, we introduce a new constraint such that the data augmentation used for the write primitives minimizes the distribution shift with the initial (baseline task) distribution. This constraint introduces a level of "interference" with the initial task, thereby limiting the channel's effective capacity. Therefore, we develop optimizations to improve the capacity in this case, including a novel ML-specific substitution based error correction protocol. We believe that the proposed modeling of the problem offers new tools to better understand and mitigate potential vulnerabilities of ML, especially in the context of increasingly large models.

Engineering the Hardware/Software Interface for Robotic Platforms - A Comparison of Applied Model Checking with Prolog and Alloy

Robotic platforms serve different use cases ranging from experiments for prototyping assistive applications up to embedded systems for realizing cyber-physical systems in various domains. We are using 1:10 scale miniature vehicles as a robotic platform to conduct research in the domain of self-driving cars and collaborative vehicle fleets. Thus, experiments with different sensors like e.g.~ultra-sonic, infrared, and rotary encoders need to be prepared and realized using our vehicle platform. For each setup, we need to configure the hardware/software interface board to handle all sensors and actors. Therefore, we need to find a specific configuration setting for each pin of the interface board that can handle our current hardware setup but which is also flexible enough to support further sensors or actors for future use cases. In this paper, we show how to model the domain of the configuration space for a hardware/software interface board to enable model checking for solving the tasks of finding any, all, and the best possible pin configuration. We present results from a formal experiment applying the declarative languages Alloy and Prolog to guide the process of engineering the hardware/software interface for robotic platforms on the example of a configuration complexity up to ten pins resulting in a configuration space greater than 14.5 million possibilities. Our results show that our domain model in Alloy performs better compared to Prolog to find feasible solutions for larger configurations with an average time of 0.58s. To find the best solution, our model for Prolog performs better taking only 1.38s for the largest desired configuration; however, this important use case is currently not covered by the existing tools for the hardware used as an example in this article.