Named Entity Recognition Based on Character-level Language Models and Attention Mechanism

Chaoju Hu ； Junchao Cheng

《International Core Journal of Engineering》 6卷1期 (2020/01) Pp. 196-201

https://doi.org/10.6919/ICJE.202001_6(1).0028

摘要

As a basic task in the field of natural language processing, named entity recognition plays an important role in text data processing tasks. Extracting features from the original text can be considered as the first step in the identification of named entities, but on this basic issue, traditional research still stays at the coarser granularity of words. Unlike traditional research, this paper focuses on finer granularity-character-level named entity recognition research. In order to fully extract the character-level feature representation from the character-level language model, this paper uses CNN and BiLSTM to perform feature extraction together, and introduces the attention mechanism to achieve more effective combination of character features and word features, then combines with BiLSTM-CRF to construct a complete end-to-end deep learning model (At- BiLSTM-CNNs-CRF). The experimental results show that its recognition ability exceeds most deep learning models.

關鍵字

Named entity recognition ； Attention mechanism ； Character-level language models ； Natural language processing

參考文獻

Yu, X., Mayhew, S., Sammons, M., & Roth, D. (2018). On the Strength of Character Language Models for Multilingual Named Entity Recognition. empirical methods in natural language processing.

Pennington, J., Socher, R., & Manning, C. D. (2014). Glove: Global Vectors for Word Representation. empirical methods in natural language processing.

Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780.

Huang, Z., Xu, W. L., & Yu, K. (2015). Bidirectional LSTM-CRF Models for Sequence Tagging. arXiv: Computation and Language,.

Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed Representations of Words and Phrases and their Compositionality. arXiv: Computation and Language,.