利用鄰居節點學習知識圖譜表示法

Translating Representations of Knowledge Graphs with Neighbors

王俊智(Chun-Chih Wang)

指導教授 ： 鄭卜壬

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2017年)

https://doi.org/10.6342/NTU201704192

摘要

在人工智慧領域當中，知識圖譜扮演著非常重要的角色，因為他們含有大量有用的資訊，不管是自然語言處理、問答系統或是搜尋引擎都被廣泛應用。然而，大部分的知識圖譜都非常的不完整。因此，找到一個有效又能處理大量資料的方法，來幫助知識圖譜補足遺失的資料則是一個非常重要的議題。
目前既有的方法，以向量表示法的模型受到最多關注，因為他們不管在準確性或是效能上都有非常傑出的表現。然而他們將所有在知識圖譜內的資料都視為獨立的個體，這部分不合理的原因在於在真實資料當中，個體之間其實存在著特定的關係。舉例來說，當我們得知某個人曾經出版過某張專輯，同時也可以確認該位是一名歌手，因此專輯和歌手並非互相獨立。
在這篇論文當中，我們提出一個除了利用節點本身以外，透過整合鄰居節點的資訊，來協助提高知識圖譜資料的準確性。首先，我們提出了一個篩選鄰居節點的方法，避免引入過多的雜訊。再者，我們利用選擇的鄰居節點，動態產生出新的向量表示法，再利用該表示法預測兩節點之間的關係。除此之外，由於所有鄰居節點都有不同程度的重要性及影響力，我們透過注意力機制來調整不同鄰居節點的權重。實驗部分，我們利用基準資料集來驗證我們的模型，不管是連結預測或是分類都有非常傑出的表現。最後，我們透過一系列分析來證實我們的模型在學習到的向量表示法有較合理的結果，顯示出在加入鄰居節點的資訊以後，大幅提升了在知識圖譜上的成效。

關鍵字

知識圖譜 ； 向量表示法 ； 多關係圖 ； 連結預測 ； 三重資料分類 ； 自然語言處理

並列摘要

In the field of AI, knowledge graphs play an important role because of their huge amount of resources. They have been applied to several tasks, such as natural language processing, question answering, and search engine. However, most knowledge graphs are far from complete. Hence, finding an efficient and effective approach is a significant issue in knowledge graphs.
Among all previous works, embedding models have caught most attention due to their performance and efficiency. Nonetheless, they consider triples in the knowledge graph independently. It is unreasonable since many triples connect to each other in reality and there exist many hidden relations between them even though they are not linked together. For example, if one person has published an album, he or she is a singer by profession. Therefore, publishing an album is highly correlated with being a singer.
In this paper, in addition to use information of an entity itself, we would like to enhance the knowledge graph completion more effectively by integrating the information provided by its neighbors. To start with, we come up with a method to select effective neighbors in order to prevent from introducing too many noises. Second, we utilize the filtered neighbors to generate the neighbor-based entity embeddings dynamically and use the created embeddings to predict the relationships between entities. Furthermore, since every neighbor has different power of influence, we exploit attention mechanism to weight neighbors according to their importance. In experiments, we implement our model on several benchmark datasets, and it outperforms other baseline methods on both link prediction and triple classification tasks. In the end, we conduct a series of analyses to justify the results produces by our model. This also demonstrates that neighborhood information is helpful on knowledge graph completion.

並列關鍵字

knowledge graph ； representation learning ； multi-relational graph ； link prediction ； triple classification ； natural language processing

參考文獻

[1] Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A Collaboratively Created Graph Database for Structuring Human Knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pages 1247-1250.

[2] George A. Miller. 1995. WordNet: A Lexical Database for English. Communications of the ACM, 28(11):39-41.

[3] Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. 2012. A Semantic Matching Energy Function for Learning with Multirelational Data. Machine Learning. 94(2): 233-259.

[4] Antione Bordes, Jason Weston, Ronan Collobert, and Toshua Bengio. 2011. Learning Structured Embeddings of Knowledge Bases. In Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, pages 301-306.

[5] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. In Advances in Neural Information Processing Systems 26, pages 2787-2795.