利用知識蒸餾於前後文相關之問題改寫以提升對話式問答

Contextual Question Rewriting with Knowledge Distillation for Improving Conversational Question Answering

黃千芝(Chien-Chi Huang)

指導教授 ： 陳縕儂

國立臺灣大學/電機資訊學院/資訊工程學系/碩士(2023年)

https://doi.org/10.6342/NTU202300619

摘要

智慧助理越來越普及的環境下，人類與機器的對話互動更趨近人與人之間的口說關係，因此，人類的對話習慣中，省略主詞或使用代名詞使得語意不完整的現象，對機器理解是一項重大的挑戰。機器必須從歷史紀錄中抽取對話中被省略的資訊，將使用者的問題重新組織成完整的問句，再依完整問句來搜尋回答。
本篇論文分析不同模型在對話式問答的問題重寫上的表現，以及在不同的資料集和不同資料特性上的表現與泛化能力等差異。並且提出一種在訓練過程中，利用知識蒸餾的技術加強模型的能力的方法，以提高改寫後的問題品質。

關鍵字

對話式問答 ； 不完整語句改寫 ； 知識蒸餾

並列摘要

The dialogue between humans and machines is more similar to the oral language with the increasing popularity of intelligent assistants. Therefore, the task of incomplete utterance rewriting (IUR) in multi-turns conversations is a major challenge for machines to understand. To answer user's questions, the machines will extract the omitted information form historical records , reconstruct user's utterances into complete questions and response to the questions. This paper analyzes the performance of different models on incomplete utterance rewriting tasks and compare the differences on generalization ability, data features and data domains. In addition, we propose a knowledge distillation techniques to strengthen the performance of the models, and improve the performance of rewritten questions.

並列關鍵字

Conversational Question Answering ； Incomplete Utterance Rewriting ； Knowledge Distillation

參考文獻

[1] E. Choi, H. He, M. Iyyer, M. Yatskar, W.-t. Yih, Y. Choi, P. Liang, and L. Zettle- moyer. Quac: Question answering in context. arXiv preprint arXiv:1808.07036, 2018.

[2] M. Del Tredici, G. Barlacchi, X. Shen, W. Cheng, and A. de Gispert. Ques- tion rewriting for open-domain conversational qa: Best practices and limitations. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 2974–2978, 2021.

[3] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.

[4] A. Elgohary, D. Peskov, and J. Boyd-Graber. Can you unpack that? learning to rewrite questions-in-context. Can You Unpack That? Learning to Rewrite Questions-in-Context, 2019.

[5] M. Grbovic, N. Djuric, V. Radosavljevic, F. Silvestri, and N. Bhamidipati. Context- and content-aware embeddings for query rewriting in sponsored search. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pages 383–392, 2015.