利用語法結構之雙向遞迴類神經網路於命名實體辨識之研究

Leveraging Linguistic Structures for Named Entity Recognition with Bidirectional Recursive Neural Networks

李朋軒(Peng-Hsuan Li)

指導教授 ： 許永真

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2017年)

https://doi.org/10.6342/NTU201704465

摘要

命名實體辨識(NER)是一個找出文字中的命名實體的重要任務，其產出能提供給下游的任務比如自然語言理解使用。此問題常從命名實體所在的文字區段的預測被轉型為線性地預測每個單詞是否屬於某一命名實體的一部分。利用CRF與RNN等模型，這類轉型後的方法取得了很好的成果。然而，每個命名實體都應該是一個語法單元，而線性單詞預測的方法忽略這個資訊。

在本論文中，我們提出一個語法導向的方法以完整利用文字裡的語言結構。要利用階層性的詞組結構，我們首先產生語法剖析樹並將之改變為最小化剖析樹與命名實體之間的不一致的語法圖。然後我們利用雙向遞迴類神經網路(BRNN)去傳遞相關的結構資訊到每一個語法單元。我們利用一個由下往上的遍歷來蒐集局部資訊，以及一個由上往下的遍歷來蒐集全域資訊。實驗顯示此方法可和線性單詞標記法相比，並在OntoNotes 5.0 NER語料上取得了超過87\% F1分數的顯著進步。

關鍵字

實體辨識 ； 類神經網路 ； 資訊抽取

並列摘要

Named Entity Recognition (NER) is an important task which locates proper names in text for downstream tasks, e.g. to facilitate natural language understanding. The problem is often casted from structured prediction of text chunks to sequential labeling of tokens. Such sequential approaches have achieved high performance with models like conditional random fields and recurrent neural networks. However, named entities should be linguistic constituents, and sequential token labeling neglects this information.

In the thesis, we propose a constituency-oriented approach which fully utilizes linguistic structures in text. First, to leverage the prior knowledge of hierarchical phrase structures, we generate parses and alter them into constituency graphs that minimize inconsistencies between parses and named entities. Then, we use Bidirectional Recursive Neural Networks (BRNN) to propagate relevant structure information to each constituent. We use a bottom-up pass to capture the local information and a top-down pass to capture the global information. Experiments show that this approach is comparable to sequential token labeling, and significant improvements can be seen on OntoNotes 5.0 NER, with F1 scores over 87\%.

並列關鍵字

Named Entity Recognition ； Neural Network ； Information Extraction

參考文獻

[2] N. Chinchor and P. Robinson. MUC-7 Named Entity Task Definition. In Proceedings of the 7th Conference on Message Understanding, volume 29, 1997.

[4] M. Collins. Head-Driven Statistical Models for Natural Language Parsing. PhD thesis, University of Pennsylvania, 1999.

[6] G. Cybenko. Approximation by Superpositions of a Sigmoidal Function. Mathematics of Control, Signals, and Systems (MCSS), 2(4):303–314, 1989.

[14] O. Irsoy and C. Cardie. Bidirectional Recursive Neural Networks for Token-Level Labeling with Structure. In NIPS Deep Learning Workshop, 2013.

[20] G. Luo, X. Huang, C.-Y. Lin, and Z. Nie. Joint Named Entity Recognition and Disambiguation. In Proc. EMNLP, pages 879–888, 2015.