利用動態階層式遞歸編碼器解碼器架構生成繁體中文對話

Using Dynamic Hierarchical Recurrent Encoder-Decoder Architecture to Generate Traditional Chinese Dialogue

林建甫(Chien-Fu Lin)

指導教授 ： 黃鐘揚

國立臺灣大學/電機資訊學院/電子工程學研究所/碩士(2017年)

https://doi.org/10.6342/NTU201703159

摘要

在對話生成的領域中，根據應用情境，將對話生成分為任務導向以及開放領域。在任務導向的對話生成模型，由於應用領域上的限制，通常使用規則導向的模型。然而，規則導向的模型具有可拓展性較差和應用範疇受到手刻特徵限制的缺點。
本研究採用端到端神經網絡模型，而非規則導向的模型，以確保模型的一般性。我們採用階層式遞歸編碼器解碼器（HRED）架構作為我們的模型，並擴展了此架構，以便能夠動態地控制生成對話的長度。我們證明由任務導向的語料庫訓練的端到端神經網絡可以產生特定領域的對話。
此外，我們還探討了HRED將對話向量化的潛力。透過HRED生成的嵌入式向量在我們的實驗中勝過其他嵌入方法。

關鍵字

對話生成 ； 階層式遞歸編碼器解碼器 ； 深度學習 ； 端到端模型 ； 繁體中文文本

並列摘要

In the domain of dialogue generating, there are two types of tasks according to application targets: task-oriented task and open-domain task. Limited by the specific usages, the task-oriented application usually adopts rule-based models. However, rule-based models are hard to extend and the applicable domains are constricted by the hand-craft features.
Instead of using the rule-based models for the task-oriented target, we apply an end-to-end neural network model trained by the task-oriented corpus to ensure the generalization of our model. We adopt the hierarchical recurrent encoder-decoder (HRED) architecture as our model and extend it to dynamically control the generated dialog’s length. This work demonstrates that the end-to-end neural network trained by task-oriented corpus could generate specific domain dialogue.
In addition, we explore the potential of the HRED on embedding the dialogue. The embedded vectors created by HRED outperforms other embedding methods on our experiment.

並列關鍵字

dialogue generating ； hierarchical recurrent encoder-decoder ； deep learning ； end-to-end model ； Traditional Chinese corpus

參考文獻

[2] Williams, Ronald J., and David Zipser. "A learning algorithm for continually running fully recurrent neural networks." Neural computation 1.2 (1989): 270-280.

[4] Press, Ofir, and Lior Wolf. "Using the output embedding to improve language models." arXiv preprint arXiv:1608.05859 (2016).

[5] Cho, Kyunghyun, et al. "Learning phrase representations using RNN encoder-decoder for statistical machine translation." arXiv preprint arXiv:1406.1078 (2014).

[9] Shang, Lifeng, Zhengdong Lu, and Hang Li. "Neural responding machine for short-text conversation." arXiv preprint arXiv:1503.02364 (2015).

[10] Yu, Lantao, et al. "SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient." AAAI. 2017.