Style Vectors for Steering Generative Large Language Model

This research explores strategies for steering the output of large language models (LLMs) towards specific styles, such as sentiment, emotion, or writing style, by adding style vectors to the activations of hidden layers during text generation. We show that style vectors can be simply computed from recorded layer activations for input texts in a specific style in contrast to more complex training-based approaches. Through a series of experiments, we demonstrate the effectiveness of activation engineering using such style vectors to influence the style of generated text in a nuanced and parameterisable way, distinguishing it from prompt engineering. The presented research constitutes a significant step towards developing more adaptive and effective AI-empowered interactive systems.

Telerobotic Mars Mission for Lava Tube Exploration and Examination of Life

The general profile and overarching goal of this proposed mission is to pioneer potentially highly beneficial (even vital) and cost-effective techniques for the future human colonization of Mars. Adopting radically new and disruptive solutions untested in the Martian context, our approach is one of high risk and high reward. The real possibility of such a solution failing has prompted us to base our mission architecture around a rover carrying a set of 6 distinct experimental payloads, each capable of operating independently on the others, thus substantially increasing the chances of the mission yielding some valuable findings. At the same time, we sought to exploit available synergies by assembling a combination of payloads that would together form a coherent experimental ecosystem, with each payload providing potential value to the others. Apart from providing such a testbed for evaluation of novel technological solutions, another aim of our proposed mission is to help generate scientific know-how enhancing our understanding of the Red Planet. To this end, our mission takes aim at the Nili-Fossae region, rich in natural resources (and carbonates in particular), past water repositories and signs of volcanic activity. With our proposed experimental payloads, we intend to explore existing lava-tubes, search for signs of past life and assess their potentially valuable geological features for future base building. We will evaluate biomatter in the form of plants and fungi as possible food and base-building materials respectively. Finally, we seek to explore a variety of novel power generation techniques using the Martian atmosphere and gravity. As detailed throughout the remainder of this chapter, this assemblage of experimental payloads, then, constitutes the backbone of our proposed telerobotic mission to Mars.

Key Information Extraction From Documents: Evaluation And Generator

Extracting information from documents usually relies on natural language processing methods working on one-dimensional sequences of text. In some cases, for example, for the extraction of key information from semi-structured documents, such as invoice-documents, spatial and formatting information of text are crucial to understand the contextual meaning. Convolutional neural networks are already common in computer vision models to process and extract relationships in multidimensional data. Therefore, natural language processing models have already been combined with computer vision models in the past, to benefit from e.g. positional information and to improve performance of these key information extraction models. Existing models were either trained on unpublished data sets or on an annotated collection of receipts, which did not focus on PDF-like documents. Hence, in this research project a template-based document generator was created to compare state-of-the-art models for information extraction. An existing information extraction model "Chargrid" (Katti et al., 2019) was reconstructed and the impact of a bounding box regression decoder, as well as the impact of an NLP pre-processing step was evaluated for information extraction from documents. The results have shown that NLP based pre-processing is beneficial for model performance. However, the use of a bounding box regression decoder increases the model performance only for fields that do not follow a rectangular shape.