Differentiating Approach and Avoidance from Traditional Notions of Sentiment in Economic Contexts

There is growing interest in the role of sentiment in economic decision-making. However, most research on the subject has focused on positive and negative valence. Conviction Narrative Theory (CNT) places Approach and Avoidance sentiment (that which drives action) at the heart of real-world decision-making, and argues that it better captures emotion in financial markets. This research, bringing together psychology and machine learning, introduces new techniques to differentiate Approach and Avoidance from positive and negative sentiment on a fundamental level of meaning. It does this by comparing word-lists, previously constructed to capture these concepts in text data, across a large range of semantic features. The results demonstrate that Avoidance in particular is well defined as a separate type of emotion, which is evaluative/cognitive and action-orientated in nature. Refining the Avoidance word-list according to these features improves macroeconomic models, suggesting that they capture the essence of Avoidance and that it plays a crucial role in driving real-world economic decision-making.

Deriving Contextualised Semantic Features from BERT (and Other Transformer Model) Embeddings

Models based on the transformer architecture, such as BERT, have marked a crucial step forward in the field of Natural Language Processing. Importantly, they allow the creation of word embeddings that capture important semantic information about words in context. However, as single entities, these embeddings are difficult to interpret and the models used to create them have been described as opaque. Binder and colleagues proposed an intuitive embedding space where each dimension is based on one of 65 core semantic features. Unfortunately, the space only exists for a small dataset of 535 words, limiting its uses. Previous work (Utsumi, 2018, 2020, Turton, Vinson & Smith, 2020) has shown that Binder features can be derived from static embeddings and successfully extrapolated to a large new vocabulary. Taking the next step, this paper demonstrates that Binder features can be derived from the BERT embedding space. This provides contextualised Binder embeddings, which can aid in understanding semantic differences between words in context. It additionally provides insights into how semantic features are represented across the different layers of the BERT model.