- 學位論文
通用序列模型：一種通用的自然語言序列到序列學習的架構
GS2S: An architecture for Generalized Natural Langauge Sequence to Sequence Learning
摘要
建立智能對話系統一直是學術界和工業界人工智能的長期目標。近年來，人們提出了各種神經網絡架構來處理序列到序列的自然語言問題，如機器翻譯、文本到結構式查詢語言、知識性問答、社交聊天機器人等。然而，現有的架構無法有效處理現實世界中覆雜的自然語言任務，例如涉及知識推理的開放領域對話。在本文中，我們介紹了一種端到端的自然語言處理模型，稱之為通用序列模型，它可以實現通用性的序列到序列學習，包含一種新穎的兩階段式架構：單個模型接收自然語言輸入，可執行所有傳統上通常由多個模塊所完成的工作，如槽填充、分類、命名實體標註、特定領域規則等，並生成適當的自然語言回覆。我們提出了包括功能式標記化和動態序列解碼在內的原創的設計和推理策略，幫助該架構實現其功能。在我們的標注數據集上進行的實驗表明，與現有的端到端的模型相比，對於牽涉知識推理的問答，我們的架構在通用性、可解釋性和可維護性方面更加有效和優越。
並列摘要
Building intelligent dialog systems has been a long-running goal of artificial intelli gence in both academic and industrial communities. In recent years, various neural net work architectures have been proposed to deal with sequence to sequence (S2S) NLP problems, such as machine translation, text to SQL, knowledge question answering, so cial chatbots, etc. However, existing architectures fail to work effectively on real-world complicated NLP tasks like open-domain conversation that involve knowledge reasoning. In this paper, we introduce a sequence to sequence learning model called GS2S, which stands for generalized sequence to sequence learning and contains a novel ”two-stage” architecture: a single model takes a natural language input, performs all of the tasks to gether that are conventionally done by multiple modules such as slot filling, classification, named entity recognition, domain specific rules, etc., and generates a proper response. In addition, novel design and inference strategies including functional tokenization and dy namic sequence decoding are proposed that help the architecture realize its capabilities. Experiment on our labeled dataset illustrates the effectiveness and superiorities of our architecture over existing end-to-end models in terms of accuracy, maintainability, inter pretability, and generalizability.