Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

"Sentiment": models, code, and papers

Fine-tuning Pretrained Multilingual BERT Model for Indonesian Aspect-based Sentiment Analysis

Mar 05, 2021
Annisa Nurul Azhar, Masayu Leylia Khodra

Although previous research on Aspect-based Sentiment Analysis (ABSA) for Indonesian reviews in hotel domain has been conducted using CNN and XGBoost, its model did not generalize well in test data and high number of OOV words contributed to misclassification cases. Nowadays, most state-of-the-art results for wide array of NLP tasks are achieved by utilizing pretrained language representation. In this paper, we intend to incorporate one of the foremost language representation model, BERT, to perform ABSA in Indonesian reviews dataset. By combining multilingual BERT (m-BERT) with task transformation method, we manage to achieve significant improvement by 8% on the F1-score compared to the result from our previous study.

  Access Paper or Ask Questions

Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysis

Sep 04, 2021
Sijie Mai, Ying Zeng, Shuangjia Zheng, Haifeng Hu

The wide application of smart devices enables the availability of multimodal data, which can be utilized in many tasks. In the field of multimodal sentiment analysis (MSA), most previous works focus on exploring intra- and inter-modal interactions. However, training a network with cross-modal information (language, visual, audio) is still challenging due to the modality gap, and existing methods still cannot ensure to sufficiently learn intra-/inter-modal dynamics. Besides, while learning dynamics within each sample draws great attention, the learning of inter-class relationships is neglected. Moreover, the size of datasets limits the generalization ability of existing methods. To address the afore-mentioned issues, we propose a novel framework HyCon for hybrid contrastive learning of tri-modal representation. Specifically, we simultaneously perform intra-/inter-modal contrastive learning and semi-contrastive learning (that is why we call it hybrid contrastive learning), with which the model can fully explore cross-modal interactions, preserve inter-class relationships and reduce the modality gap. Besides, a refinement term is devised to prevent the model falling into a sub-optimal solution. Moreover, HyCon can naturally generate a large amount of training pairs for better generalization and reduce the negative effect of limited datasets. Extensive experiments on public datasets demonstrate that our proposed method outperforms existing works.

* Under Review 

  Access Paper or Ask Questions

WESSA at SemEval-2020 Task 9: Code-Mixed Sentiment Analysis using Transformers

Sep 21, 2020
Ahmed Sultan, Mahmoud Salim, Amina Gaber, Islam El Hosary

In this paper, we describe our system submitted for SemEval 2020 Task 9, Sentiment Analysis for Code-Mixed Social Media Text alongside other experiments. Our best performing system is a Transfer Learning-based model that fine-tunes "XLM-RoBERTa", a transformer-based multilingual masked language model, on monolingual English and Spanish data and Spanish-English code-mixed data. Our system outperforms the official task baseline by achieving a 70.1% average F1-Score on the official leaderboard using the test set. For later submissions, our system manages to achieve a 75.9% average F1-Score on the test set using CodaLab username "ahmed0sultan".

* Proceedings of SemEval-2020 

  Access Paper or Ask Questions

PGCD: a position-guied contributive distribution unit for aspect based sentiment analysis

Aug 11, 2021
Zijian Zhang, Chenxin Zhang, Qin Liu, Hongming Zhu, Jiangfeng Li

Aspect based sentiment analysis (ABSA), exploring sentim- ent polarity of aspect-given sentence, has drawn widespread applications in social media and public opinion. Previously researches typically derive aspect-independent representation by sentence feature generation only depending on text data. In this paper, we propose a Position-Guided Contributive Distribution (PGCD) unit. It achieves a position-dependent contributive pattern and generates aspect-related statement feature for ABSA task. Quoted from Shapley Value, PGCD can gain position-guided contextual contribution and enhance the aspect-based representation. Furthermore, the unit can be used for improving effects on multimodal ABSA task, whose datasets restructured by ourselves. Extensive experiments on both text and text-audio level using dataset (SemEval) show that by applying the proposed unit, the mainstream models advance performance in accuracy and F1 score.

* ICASSP 2021 

  Access Paper or Ask Questions

Cross-lingual Transfer of Twitter Sentiment Models Using a Common Vector Space

May 18, 2020
Marko Robnik-Sikonja, Kristjan Reba, Igor Mozetic

Word embeddings represent words in a numeric space in such a way that semantic relations between words are encoded as distances and directions in the vector space. Cross-lingual word embeddings map words from one language to the vector space of another language, or words from multiple languages to the same vector space where similar words are aligned. Cross-lingual embeddings can be used to transfer machine learning models between languages and thereby compensate for insufficient data in less-resourced languages. We use cross-lingual word embeddings to transfer machine learning prediction models for Twitter sentiment between 13 languages. We focus on two transfer mechanisms using the joint numerical space for many languages as implemented in the LASER library: the transfer of trained models, and expansion of training sets with instances from other languages. Our experiments show that the transfer of models between similar languages is sensible, while dataset expansion did not increase the predictive performance.

  Access Paper or Ask Questions

Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales

Jun 17, 2005
Bo Pang, Lillian Lee

We address the rating-inference problem, wherein rather than simply decide whether a review is "thumbs up" or "thumbs down", as in previous sentiment analysis work, one must determine an author's evaluation with respect to a multi-point scale (e.g., one to five "stars"). This task represents an interesting twist on standard multi-class text categorization because there are several different degrees of similarity between class labels; for example, "three stars" is intuitively closer to "four stars" than to "one star". We first evaluate human performance at the task. Then, we apply a meta-algorithm, based on a metric labeling formulation of the problem, that alters a given n-ary classifier's output in an explicit attempt to ensure that similar items receive similar labels. We show that the meta-algorithm can provide significant improvements over both multi-class and regression versions of SVMs when we employ a novel similarity measure appropriate to the problem.

* To appear, Proceedings of ACL 2005 

  Access Paper or Ask Questions

The Power of Language: Understanding Sentiment Towards the Climate Emergency using Twitter Data

Jan 25, 2021
Arman Sarjou

Understanding how attitudes towards the Climate Emergency vary can hold the key to driving policy changes for effective action to mitigate climate related risk. The Oil and Gas industry account for a significant proportion of global emissions and so it could be speculated that there is a relationship between Crude Oil Futures and sentiment towards the Climate Emergency. Using Latent Dirichlet Allocation for Topic Modelling on a bespoke Twitter dataset, this study shows that it is possible to split the conversation surrounding the Climate Emergency into 3 distinct topics. Forecasting Crude Oil Futures using Seasonal AutoRegressive Integrated Moving Average Modelling gives promising results with a root mean squared error of 0.196 and 0.209 on the training and testing data respectively. Understanding variation in attitudes towards climate emergency provides inconclusive results which could be improved using spatial-temporal analysis methods such as Density Based Clustering (DBSCAN).

* 6 Pages, 10 figures 

  Access Paper or Ask Questions

Amobee at SemEval-2018 Task 1: GRU Neural Network with a CNN Attention Mechanism for Sentiment Classification

Apr 12, 2018
Alon Rozental, Daniel Fleischer

This paper describes the participation of Amobee in the shared sentiment analysis task at SemEval 2018. We participated in all the English sub-tasks and the Spanish valence tasks. Our system consists of three parts: training task-specific word embeddings, training a model consisting of gated-recurrent-units (GRU) with a convolution neural network (CNN) attention mechanism and training stacking-based ensembles for each of the sub-tasks. Our algorithm reached 3rd and 1st places in the valence ordinal classification sub-tasks in English and Spanish, respectively.

* 8 pages, accepted to the 12th International Workshop on Semantic Evaluation 2018 

  Access Paper or Ask Questions

Sentiment Analysis Using Simplified Long Short-term Memory Recurrent Neural Networks

May 08, 2020
Karthik Gopalakrishnan, Fathi M. Salem

LSTM or Long Short Term Memory Networks is a specific type of Recurrent Neural Network (RNN) that is very effective in dealing with long sequence data and learning long term dependencies. In this work, we perform sentiment analysis on a GOP Debate Twitter dataset. To speed up training and reduce the computational cost and time, six different parameter reduced slim versions of the LSTM model (slim LSTM) are proposed. We evaluate two of these models on the dataset. The performance of these two LSTM models along with the standard LSTM model is compared. The effect of Bidirectional LSTM Layers is also studied. The work also consists of a study to choose the best architecture, apart from establishing the best set of hyper parameters for different LSTM Models.

* 6 pages, 6 figures, 6 tables 

  Access Paper or Ask Questions

Graph Ensemble Learning over Multiple Dependency Trees for Aspect-level Sentiment Classification

Mar 12, 2021
Xiaochen Hou, Peng Qi, Guangtao Wang, Rex Ying, Jing Huang, Xiaodong He, Bowen Zhou

Recent work on aspect-level sentiment classification has demonstrated the efficacy of incorporating syntactic structures such as dependency trees with graph neural networks(GNN), but these approaches are usually vulnerable to parsing errors. To better leverage syntactic information in the face of unavoidable errors, we propose a simple yet effective graph ensemble technique, GraphMerge, to make use of the predictions from differ-ent parsers. Instead of assigning one set of model parameters to each dependency tree, we first combine the dependency relations from different parses before applying GNNs over the resulting graph. This allows GNN mod-els to be robust to parse errors at no additional computational cost, and helps avoid overparameterization and overfitting from GNN layer stacking by introducing more connectivity into the ensemble graph. Our experiments on the SemEval 2014 Task 4 and ACL 14 Twitter datasets show that our GraphMerge model not only outperforms models with single dependency tree, but also beats other ensemble mod-els without adding model parameters.

* Accepted by NAACL 2021 

  Access Paper or Ask Questions