語意一致地自動生成中文新聞的標題

Semantically Consistent Title Generation for Chinese News Articles

吳政軒(Cheng-Hsuan Wu)

指導教授 ： 鄭卜壬

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2019年)

https://doi.org/10.6342/NTU201901724

摘要

在過去幾年裡，很多類神經網路模型，像是各種sequence-to-sequence的變形在標題生成這方面有了很大的進展。Pointer-Generator就是其中一個具有代表性的模型，在機器翻譯、概要、標題生成等等自然語言處理的任務都有不錯的表現。然而，這些模型常常有一個問題，就是生成標題的語意和文章重點的語意不太一致，尤其在中文的任務更是明顯。在這篇論文裡，為了解決上述的問題，我們提出兩個方法加在Pointer-Generator模型上。第一，我們藉由告知模型上一個時間點的上下文語意，改善了注意力機制，讓模型在掌握當前部分的語意後可以注意到其他部分，避免再繼續抄整個句子，那些句子裡可能含有一些比較不重要的資訊。第二，我們針對編碼器加了一些限制，讓他對文章編碼時，可以保有較多的語意資訊。我們將提出的模型應用在兩個台灣新聞報社的中文資料集，結果顯示我們的模型不管是在ROUGE分數上或是人類評估的結果，表現都比Pointer-Generator好。

關鍵字

標題生成 ； 自然語言處理 ； 語意一致性 ； 注意力機制 ； Pointer-Generator

並列摘要

In the past years, various neural sequence-to-sequence models for title generation have made considerable progresses. Pointer-Generator is one of the representative models recently on many natural language processing tasks, such as machine translation, summarization, title generation and so on. However, these models usually suffer from semantic inconsistency between a generated title and the article, especially for Chinese title generation. In this paper, we propose two methods which augment Pointer-Generator to tackle the issue. First, we improve the attention mechanism by informing model the previous context semantic. By doing so, our model can attend to other parts when it has already got the meanings of the current part. It can prevent model from continuous coping the whole sentence, which probably contains less important information. Second, we add a constraint to the encoder so that it can encode the article with more important semantic information. We apply our model to the two Chinese news datasets collected from two newspaper offices in Taiwan, outperforming Pointer-Generator on both ROUGE scores and the human evaluation.

並列關鍵字

Title Generation ； Natural Language Processing ； Semantic Consistent ； Attention Mechanism ； Pointer-Generator

參考文獻

[1] Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073-1083, Vancouver, Canada. Association for Computational Linguistics.

[2] Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2017. SummaRuNNer: A recurrent neural network based sequence model for extractive summarization of documents. In Association for the Advancement of Artificial Intelligence.

[3] Ziqiang Cao, Furu, Wei, Wenjie Li, and Sujian Li. 2017b. Faithful to the Original: Fact aware neural abstractive summarization. arXiv preprint arXiv:1711.04434.

[4] Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018a. Ranking Sentences for Extractive Summarization with Reinforcement Learning. In Proceedings of the NAACL 2018 – Conference of the North American Chapter of the Association for Computational Linguistics.

[5] Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 379-389, Lisbon, Portugal. Association for Computational Linguistics.