注意力機制之於格狀架構式自然語言界面

Grid Structure Attention of Natural Language Interface

康家瑋(Jia-Wei Kang)

指導教授 ： 王勝德

國立臺灣大學/電機資訊學院/電機工程學研究所/碩士(2020年)

https://doi.org/10.6342/NTU201902099

摘要

自然語言界面是一種人類通過使用自然語言與計算機交互溝通的用戶界面。而近年來將自然語言翻譯為bash指令是一種較為新穎的自然語言界面問題。由於bash命令的應用領域範圍從系統管理，網絡管理，文件處理等，我們需要在這個廣泛的應用領域上進行參數化和正規化的新機制。在本文中，我們提出了一種稱為GSAM（格狀架構注意機制）的新機制，該機制用於序列到序列模型的遞歸神經網絡（RNN）基礎。我們的機制首先使用非線性函數將自然語言參數化，接著使用另一個模型來捕獲參數化過後的自然語言鄰接信息。我們通過減少訓練階段的一些重要訊息來評估GSAM的語法分數和命令準確性，並且它仍然有著了顯著的正確率。

關鍵字

人機界面 ； 自然語言處理 ； 自然語言界面 ； 語法分析 ； 注意力機制

並列摘要

Natural language interfaces are user interfaces that allow human to interact with the computer by using natural languages. Mapping natural languages to bash commands is a novel interface problem and has been attracted great curiosity in recent years. Due to application domains of bash commands including the file system administration, the networking management, the file processing, etc, we need novel mechanisms of parameterization and normalization on this rich domain. In this paper, we propose a new mechanism, called GSAM(Grid Structure Attention Mechanism). This mechanism is used in the recurrent neural networks (RNNs) base of the sequence-to-sequence model. Our mechanism first uses a non-linear function to parameterize natural language sentences, and then use another model to capture the adjacency information of the natural language sentences. To show the robustness of GSAM, we reduce the training dataset of utility bash commands. GSAM still get better performance in Template Matching scores and BLEU-scores.

並列關鍵字

human-computer interaction ； natural language processing ； natural language interface ； semantic parsing ； attention mechanism

參考文獻

[1] D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.

[2] P. F. Brown, P. V. Desouza, R. L. Mercer, V. J. D. Pietra, and J. C. Lai. Class-based n-gram models of natural language. Computational linguistics, 18(4):467–479, 1992.

[3] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555,2014.

[4] W. Cui, Y. Xiao, H. Wang, Y. Song, S.-w. Hwang, and W. Wang. Kbqa: learning question answering over qa corpora and knowledge bases. arXiv preprint arXiv:1903.02419, 2019.

[5] E. W. Dijkstra. On the foolishness of” natural language programming”. In Program construction, pages 51–53. Springer, 1979.