Robert_LSTM Short Text Sentiment Classification Model Based on Attention Mechanism

Shixiong Ruan ； Junbo Gao ； Jin Xiang

《International Core Journal of Engineering》 7卷3期 (2021/03) Pp. 213-219

https://doi.org/10.6919/ICJE.202103_7(3).0030

摘要

Microblog, as an important platform for people to express their opinions, has become a research hotspot of text sentiment analysis. There are some problems in micro-blog texts, such as casual expression, mixed Chinese and English, and difficult identification of emotional words on the Internet, which make it difficult to determine emotion classification. This paper proposes a BERT_LSTM model based on attention mechanism. Firstly, the Roberta model is used to extract semantic features and input them into the LSTM module, and then the semantic features are transferred repeatedly and finally input into the Attention layer to obtain weighted features. Finally, Softmax linear function is used to achieve sentiment classification. The comparative experimental results show that the accuracy of Roberta_LSTM model based on attention mechanism proposed in this paper is 2.1% higher than that of single-language Roberta model, which shows that this model can more accurately capture the emotional tendency of microblog short texts.

關鍵字

Attentional Mechanism ； RoBERTa Model ； LSTM Model ； Softmax Linear Functions ； Emotion Classification

參考文獻

Ouyang X, Zhou P, Li C H, et al. Sentiment analysis using convolutional neural network [C]// IEEE International Conference on Computer & Information Technology, Ubiquitous Computing & Communications, Dependable, IEEE, 2015: 2359-2364.

Wang W, Pan S J, Dahlmeier D, et al.Recursive neural conditional random fields for aspect-based sentiment analysis [C]// Conference on Empirical Method in Nature Language. 2016.

Pappas N, Popescu-Belis A. Multilingual hierarchical attention networks for document classification [C]// Peoceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2017: 1015-1025.

Meng X, Wei F, Liu X, et al.Cross-lingual mixture model for sentiment classification [C]// Meeting of the Association for Computational Linguistics: Long Papers. 2013.

Li Yuqing, Li Xin, Han Xu, et al. Sentiment Analysis of Microblog Based on Bilingual Dictionary [J]. Acta Electronica Sinica, 2016,44 (9): 2068-2073.