不平衡資料集應用於問答系統答案驗證之研究

A Study on Imbalanced Dataset of Answer Validation for Question Answering System

蔡承家(Cheng-Chia Tsai)

指導教授 ： 戴敏育

淡江大學/商管學院/資訊管理學系碩士班/碩士(2016年)

https://doi.org/10.6846/TKU.2016.00577

摘要

問答系統(Question answering)主要是在解決給定一道問題，透過機器閱讀(Machine Reading)的方式讓系統能夠理解這一道題目後進行回答。問答系統通常包含了問題分析(Question Analysis)、文件檢索(Document Retrieval)、答案抽取(Answer Extraction)、答案驗證(Answer Validation)。
　　在過去文獻中有相當多的問答系統相關研究，但是並未對問答系統中答案驗證不平衡資料集與平衡資料集進行深入探討。本研究目的會透過機器學習完整分析不平衡資料集與平衡資料集。
　　本研究使用 NTCIR-12 QA-Lab2 日本大學入學考試世界歷史資料集，此資料集與以往問答系統比較不同的地方在於是系統必須先理解一篇短文之後，才能夠回應接下來相關的問題。
　　本研究針對不平衡資料集與平衡資料集提出了許多的模型，藉由最佳化參數與交叉驗證後，實驗結果顯示在不平衡資料集中，最佳模型的正確率達到了 90%。本論文主要貢獻為提出了一套問答系統，並且在答案驗證階段透過不平衡資料集與平衡資料集證實，不平衡資料集所建構出來之模型顯著性較高。

關鍵字

機器學習 ； 不平衡資料集 ； 問答系統 ； 支持向量機器 ； 答案驗證 ； 大學考試 ； QA-Lab

並列摘要

Question Answering is a system that can process and answer a given question. Question Answering system usually consists of four stages: Question Analysis, Document Retrieval, Answer Extraction and Answer Validation.
Although a considerable number of studies have been made on Question Answering system, little is known about the power of Imbalanced datasets and balanced datasets for Answer Validation from Question Answering.
The purpose of this paper is to provide a comprehensive analysis of Imbalanced datasets and balanced datasets through machine learning.
In this paper, we used datasets from NTCIR-12 QA-Lab2 Japanese university entrance exams on the subject of "World History". The difference between this datasets and previous ones lies in the different processing method that the system needed to understand a context provided by the present research’s datasets and answered the following related questions.
The study presented many Imbalanced datasets and Balanced datasets models by using f.select and Cross Validation. The results show the best performance of our system achieved an accuracy of 90% in the Imbalanced datasets model.
The main contribution of this study was in proposing a question answering system for Japanese university entrance exams and providing evidence that the Imbalanced datasets model outperformed the balanced datasets model for Answer Validation.

並列關鍵字

Machine learning ； Imbalanced Dataset ； Question Answering ； Support Vector Machine ； Answer Validation ； University entrance examination ； QA-Lab

參考文獻

[13] Magnini, B., Negri, M., Prevete, R., & Tanev, H. (2002). Is it the right answer?: Exploiting web redundancy for answer validation. Paper presented at the Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, 425-432.

[29] Shima, H., Kanayama, H., Lee, C., Lin, C., Mitamura, T., Miyao, Y., . . . Takeda, K. (2011). Overview of ntcir-9 rite: Recognizing inference in text. Paper presented at the Proceedings of the 9th NII Test Collection for Information Retrieval Workshop (NTCIR’11), 291-301.

[2] Anand, A., Pugalenthi, G., Fogel, G. B., & Suganthan, P. (2010a). An approach for classification of highly imbalanced data using weighting and undersampling. Amino Acids, 39(5), 1385-1391.

[3] Anand, A., Pugalenthi, G., Fogel, G. B., & Suganthan, P. (2010b). An approach for classification of highly imbalanced data using weighting and undersampling. Amino Acids, 39(5), 1385-1391.

[4] Batuwita, R., & Palade, V. (2012). Adjusted geometric-mean: A novel performance measure for imbalanced bioinformatics datasets learning. Journal of Bioinformatics and Computational Biology, 10(04), 1250003.