以有限配對資料訓練事實問題生成模型之研究

Fatual Question Generation Model Construction With Limited Paired Training Data

蕭雅方(Hsiao Ya-Fang)

指導教授 ： 柯佳伶

國立臺灣師範大學/理學院/資訊工程學系/碩士(2020年)

https://doi.org/10.6345/NTNU202001441

摘要

本論文考慮在閱讀文句與對應問題的配對資料有限情況下，透過遷移式學習概念，利用未配對的資料增強編碼器-解碼器架構模型的學習效果，使模型仍能生成相當於輸入大量配對資料訓練後的生成效果。本研究採用序列對序列模型，先以非監督式學習方式，利用大量無需經過標記的文句和問題，訓練自動編碼器架構。接著，擷取出預訓練好能理解文句的編碼器及生成問題的解碼器進行組合，並對編碼器加入轉移層建構出新的模型，再以遷移式學習選用文句與問題配對訓練微調模型參數。實驗結果顯示，採用本論文設計的遷移式學習方式，並配合訓練策略，在減少一半文句與問題配對資料的訓練，仍比直接採用全部配對訓練資料進行訓練得到的問題生成模型有更佳效果。

關鍵字

問題生成 ； 深度學習 ； 自然語言處理 ； 語言模型 ； 遷移學習

並列摘要

In real applications, there is usually not a large number of sentence and question pairs for training a question generation model. To solve the problem, we adopt the network-based transfer learning by using unpaired training data to enhance the learning effect of the encoder-decoder model. Accordingly, the obtained model still achieves the similar generation effect by comparing with the model which is directly trained by a large amount of paired data. In this study, we using a large number of sentences and questions that do not need to be labeled as pairs to train two auto-encoders, respectively. Then we combine the pre-trained encoder which encodes the semantics of sentence and the pre-trained decoder which generates the question. Next, by inserting a transfer layer to the encoder and fine-tune the model parameters by a fewer number of paired data. The results of experiments show that, by applying the designed training strategies, the question generation model trained by less than half of the paired training data still achieves a better performance than the model directly trained by using all the training data.

並列關鍵字

Question Generation ； Deep Learning ； Natural Language Processing ； Language Model ； Transfer Learning

參考文獻

[1] D. Bahdanau, K. Cho and Y. Bengio, "Neural Machine Translation by Jointly Learning to Align and Translate", International Conference on Learning Representations, 2015.

[2] K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor, “Freebase: a collaboratively created graph database for structuring human knowledge,” in Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247–1250, 2008.

[3] Y. Chali and S. A. Hasan, “Towards topic-to-question generation, ” Computational Linguistics, vol. 41, pp. 1-20, 2015.

[4] J. Chung, C. Gulcehre, K. Cho and Y. Bengio, Empirical evaluation of gated recurrent neural networks on sequence modeling, [online] Available: http://arxiv.org/abs/1412.3555, Dec. 2014.

[5] Y. A. Chung, H. Y. Lee, and J. Glass, “Supervised and Unsupervised Transfer Learning for Question Answering,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1, 2018.