透過改寫產生閱讀理解選擇題

Generating Multiple-Choice Reading Comprehension Questions using Paraphrase

曾雅敏(Ya-Min Tseng)

指導教授 ： 孫雅麗

國立臺灣大學/管理學院/資訊管理學研究所/碩士(2014年)

https://doi.org/10.6342/NTU.2014.02276

摘要

隨著線上英語學習環境的普及，以英語為外語（English as a Foreign Language）學習者擁有更多學習的管道，這些自主學習者需要借助自動評量（automatic assessment）工具以評估閱讀理解與學習成效。然而，現有自動出題系統（automatic question generation system）僅注重將直述句轉換為問句的過程，用字遣詞與原始文字完全相同，即使沒有正確理解篇章，也能透過搜尋文章中相同字段來判斷答案。對此，我們提出將自動改寫（paraphrase generation）結合自動出題以產生閱讀理解選項敘述。我們的系統包含三個部份，在Choice Generation System中，我們的轉換規則設計得以確保所有自動產生的選項背後都有特定的測驗目標，此外，語句關係（discourse relation）辨識和自動改寫概念的引入，可使選項更加多樣； Paraphrase Generation System透過詞彙、句構和指代的自動改寫加大選項的字面差異，並使用自動出題特有的改寫資源－名詞指代（nominal coreference），將篇章的指代關係融入改寫當中； Acceptability Ranker 使用與自動改寫領域和自動出題領域相關的特徵（feature）進行訓練，並以選項接受度為基準排序改寫選項。在最終評估中，我們的系統所產生的選項雖然在文法性、合理性和整體品質上些微地低於標竿系統，但是我們擁有較高的選項難度，並且在比較選項與來源文字的同異時，未更動的比例明顯低於標竿系統。

關鍵字

自動出題 ； 自動評量 ； 閱讀理解 ； 線上學習 ； 選擇題 ； 改寫生成 ； 語句關係

並列摘要

As online English learning environment becomes more and more ubiquitous, English as a Foreign Language (EFL) learners have more choices to learning English. There is thus an increasing demand for automatic assessment tools that help self-motivated learners evaluate their understanding and comprehension. Existing question generation systems, however, focus on the sentence-to-question surface transformation and the questions could be simply answered by word matching, even without good comprehension. We propose a novel approach to generating multiple-choice reading comprehension questions by combining paraphrase generation with question generation. To achieve this, we build a system that consists of three components. In the Choice Generation System, transformation rules are designed to ensure that every generated statement is bound up with a specific testing purpose. Discourse relation recognition and the concept of paraphrasing are also introduced into the system, enriching the choice candidates. The Paraphrase Generation System then moves on to enlarge the superficial difference by paraphrasing lexically, syntactically and referentially. We adopt QG-specific paraphrase resource, nominal coreference, into the system to capture article-wide coreferential relations. Finally, the Acceptability Ranker is trained based on useful features that have been seen in paraphrase generation and question generation to rank paraphrases by their acceptability as question choices. In the final evaluation, although there is a slight decrease in the scores of grammaticality, make-sense and overall quality, our results outperform the baseline system in the challenging score and have a significantly smaller percentage of statements that are identical to the sources sentences.

並列關鍵字

question generation ； automatic assessment ； reading comprehension ； e-learning ； multiple choice questions ； paraphrase generation ； discourse relation

參考文獻

[3] Allessi, S. M., Anderson, R. C., Anderson, T. H., Biddle, W. B., & Dalgaard, B. R. (1974). Development and Implementation of the Computer Assisted Instruction Study Management System (CAISMS): DTIC Document.

[4] Anderson, R. C., & Biddle, W. B. (1975). On asking people questions about what they are reading. Psychology of learning and motivation, 9, 89-132.

[8] Bernhard, D., De Viron, L., Moriceau, V., & Tannier, X. (2012). Question Generation for French: Collating Parsers and Paraphrasing Questions. Dialogue & Discourse, 3(2), 43-74.

[9] Bhagat, R., & Hovy, E. (2013). What is a paraphrase? Computational Linguistics.

[19] Heilman, M. (2011). Automatic factual question generation from text. Carnegie Mellon University.