A FastText Classification Model Based on Simbert Data Augmentation

Haitao Wang ； Jiaxing Fan

《World Scientific Research Journal》 9卷1期 (2023/01) Pp. 30-41

https://doi.org/10.6911/WSRJ.202301_9(1).0004

摘要

FastText is a text classification and word training tool launched by Facebook. Its biggest feature is that it greatly reduces the classification time while ensuring accuracy. However, when the training set is too small, the FastText classification model is prone to overfitting during the classification process, resulting in a decline in the classification accuracy. Data augmentation technology can effectively expand the size of the training set using existing data without introducing external data, thereby improve the performance of the text classification model and solve the problem of overfitting of the classification model caused by too small samples or uneven sample distribution. This paper proposed a data augmentation algorithm based on Simbert to improve the performance of the FastText classification model. Firstly, use the Simbert generative model to augment the sample. Then calculate the similarity between the original sample and the generated sample, and select the top K most similar samples. Finally, the original sample and the generated sample are merged into a new training set as the input of the classification model for classification. Experimental results show that FastText classification model performance improved on four publicly available datasets.

關鍵字

Data augmentation ； Text classification ； FastText ； Simbert

參考文獻

Ye, H. J., & Kankanhalli, A. (2017). Solvers’ participation in crowdsourcing platforms: Examining the impacts of trust, and benefit and cost factors. The Journal of Strategic Information Systems, 26, 101–117.

Yao T , Zhai Z , Gao B . Text Classification Model Based on fastText[C]// 2020 IEEE International Conference on Artificial Intelligence and Information Systems (ICAIIS). IEEE, 2020.

Yoon Kim. 2014. Convolutional neural networks for sentence classification. CoRR, abs/1408.5882.

Lai S , Xu L , Liu K , et al. Recurrent Convolutional Neural Networks for Text Classification. 2015.

Pan, S.J.; Yang, Q. A survey on transfer learning. IEEE T rans. Knowl. Data Eng. 2009, 22, 1345–1359. [CrossRef].