資訊擷取與知識注入技術於機器閱讀理解之研究

A Study on Information Extraction and Knowledge Injection for Machine Reading Comprehension

吳沁穎(Wu Chin-Ying)

指導教授 ： 陳柏琳

國立臺灣師範大學/理學院/資訊工程學系/碩士(2022年)

https://doi.org/10.6345/NTNU202201233

摘要

近年來，預訓練上下文語言模型 (Pre-trained Contextualized Language Modeling, PCLM) 的出現，使得基於 PCLM 的方法在各種機器閱讀理解 (Machine Reading Comprehension, MRC) 與對話式機器閱讀理解 (Conversational MRC, CMRC) 都有非常優秀的表現。然而，在機器閱讀理解領域仍然較少研究琢磨於開放領域知識 (Open-domain Knowledge) 與域內知識 (In-domain Knowledge) 的運用。有鑑於此，本論文提出一種針對MRC與CMRC的有效建模方法。此方法具有兩個主要的特點：首先，針對文章段落進行訊息提取 (Information Extraction, IE) 的預處理，藉此將每個文章段落聚類成一個偽類 (Pseudo-class) 以提供PCLM 進行訊息增強，進而提升後續 MRC與CMRC的任務表現；另一方面，本論文提出了一種新的知識注入 (Knowledge Injection, KI) 方法，將開放領域知識 (Open-domain Knowledge) 與域內知識 (In-domain Knowledge) 注入至 PCLM ，藉此捕捉更為精準的問題與文章段落間的相互關係。本論文將實驗結果與數個當今最佳的方法進行比較，除了在多個MRC與CMRC資料集上都有一定程度的表現外，大量的實證實驗也證明了本論文方法的有效性與可行性。

關鍵字

機器閱讀理解 ； 自然語言處理 ； 知識圖譜 ； 深度學習

並列摘要

In the recent past, pre-trained contextualized language modeling (PCLM) approaches have made inroads into diverse tasks of machine reading comprehension (MRC), as well as conversational MRC (CMRC), with good promise. Despite the success of these approaches, there are still not many efforts on the integration of either open-domain or in-domain knowledge into MRC and CMRC. In view of this, we propose in this thesis an effective modeling method for MRC and CMRC, which has at least two distinctive characteristics. On one hand, an information extraction (IE) preprocess is conducted to Cluster each paragraph of interest into a pseudo-class for the purpose to provide augmented information for PCLM to enhance MRC and CMRC performance. On the other hand, we also explore a novel infusion of both open-domain and in-domain knowledge into PCLM to better capture the interrelationship between a posed question and a paragraph of interest. An extensive set of empirical experiments carried out on several MRC and CMRC benchmark datasets indeed demonstrate the effectiveness and practical feasibility our proposed approach in comparison to some top-of-the-line methods.

並列關鍵字

Machine Reading Comprehension ； Natural Language Processing ； Knowledge Graph ； Deep Learning

參考文獻

[1] T. Young, D. Hazarika, S. Poria and E. Cambria, "Recent Trends in Deep Learning Based Natural Language Processing," ieee Computational intelligenCe magazine 13.3:55-75, 2018.

[2] C. Zeng, S. Li, Q. Li, J. Hu and J. Hu, "A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasets," arXiv:2006.11880, 2020.

[3] W. G. Lehnert. “The Process of Question Answering,” PhD thesis, AAI7728146, 1977.

[4] E. Riloff and M. Thelen. "A Rule-Based Question Answering System for Reading Comprehension Tests," in Proceedings of the 2000 ANLP/NAACL Workshop on Reading Comprehension Tests as Evaluation for Computer-Based Language Understanding Systems - Volume 6; Association for Computational Linguistics: USA; ANLP/NAACL-ReadingComp ’00, p. 13–19. doi:10.3115/1117595.1117598, 2000.

[5] E. Charniak, Y. Altun, R. S. Braz, B. Garrett, M. Kosmala, T. Moscovich, L. Pang, C. Pyo, Y. Sun, W. Wy, Z. Yang, S. Zeiler, and L. Zorn. "Reading Comprehension Programs in a Statistical-Language-Processing Class, " in ANLP-NAACL 2000 Workshop: Reading Comprehension Tests as Evaluation for Computer-Based Language Understanding Systems, 2000.