異質資訊網路中基於決策樹模型增強之圖神經網路

Boosting Graph Neural Network with Decision Tree-based Models for Heterogeneous Information Networks

張仕諺(Shih-Yen Chang)

指導教授 ： 林澤

國立臺灣大學/電機資訊學院/電信工程學研究所/碩士(2022年)

https://doi.org/10.6342/NTU202202860

摘要

圖神經網路(graph neural network; GNN)可萃取圖結構資訊，在眾多相關的應用場景下取得亮眼的表現，因此近年間收穫大量的關注，是新興的深度機器學習模型。然而在現實場景中，社群之間的複雜互動往往形成相異的節點與邊類型，以及目標節點的高維度的豐富資訊，因此形成異質資訊網路(heterogeneous information network; HIN)。
雖然針對異質圖的異質圖神經網路（heterogeneous graph neural network; HGNN)已有諸多研究，但傳統的異質圖神經網路模型更多地關注圖結構而非節點特徵，且仍面臨圖卷積層數少提取不足；多則表現欠佳的窘境。面對較為注重節點特徵資訊的節點分類任務，HGNN目前仍缺乏更有效的節點特徵萃取機制。
為改善異質圖神經網路面臨的問題，本論文提出一種用於節點分類任務的新穎 HGNN 模型——基於樹模型增強之異質圖神經網路(tree boosted heterogeneous graph neural network; TreeXGNN )。此模型透過整合 梯度提升決策樹(gradient boosted decision tree; GBDT)、具有共享特徵空間的HGNN、特徵融合，與多任務學習模組，可有效提升 HGNN於節點分類任務之特徵萃取，並進一步降低模型複雜度且提高模型泛化能力。
我們在三個著名的異質圖數據集 IMDB、DBLP 和 ACM 上實現最優異的性能，相比先前研究顯著提高了 3.2%。
透過實際場景的視角切入，對本模型中模組與參數探討，配合資料集的特徵可解釋性，本論文成功驗證整合決策樹模型以進行特徵萃取對於HGNN在節點分類任務中的正面意義，奠定了未來相關研究的基礎。

關鍵字

異質資訊網路 ； 圖神經網路 ； 梯度提升決策樹 ； 節點分類 ； 特徵萃取

並列摘要

Graph neural network (GNN) is an emerging deep learning model. Through graph structure information extraction and outstanding performance in many related application scenarios, GNN has received much attention. However, in real-world scenarios, a heterogeneous information network (HIN), also known as a heterogeneous graph, is often employed to describe intricate interactions between communities through the distinct node and edge types, as well as high dimensional node features providing rich information about the target nodes.
There have been many studies on the heterogeneous graph neural network (HGNN) though, the conventional HGNN models more focus on graph structure rather than node features. Moreover, HGNNs were caught in a dilemma, in which the little number of graph convolution layers brings insufficient extraction, whereas deep layers result in poor performance. In the face of node classification tasks that more concentrate on node feature information, HGNNs require a more functional node feature extraction mechanism.
Towards ameliorating the problems faced by heterogeneous graph neural networks, we propose a novel HGNN node classification model, the tree boosted heterogeneous graph neural network abbreviated as TreeXGNN. With the modules of gradient boosted decision tree (GBDT), HGNN with shared feature space, feature fusion, and joint learning, TreeXGNN can improve the feature extraction of HGNN in node classification tasks and further reduce the model complexity.
We achieve state-of-the-art performance on three well-known HIN datasets, IMDB, DBLP, and ACM, with a significant 3.2% improvement over previous literature.
From the perspective of authentic scenarios, we analyze the modules and parameters in our proposed framework. Incorporating the feature interpretability of experimental datasets, we successfully verify the positive significance of feature extraction for HGNN in the node classification task and lays a foundation for future related research.

並列關鍵字

heterogeneous information ntwork ； graph neural network ； gradient boosted decision tree ； node classification ； feature extraction

參考文獻

[1] S. Zheng, Y. Li, S. Chen, J. Xu, and Y. Yang, “Predicting drug–protein interaction using quasi­visual question answering system,” Nature Machine Intelligence, vol. 2, no. 2, pp. 134–140, 2020.

[2] M. Karimi, D. Wu, Z. Wang, and Y. Shen, “Deepaffinity: interpretable deep learn­ing of compound–protein affinity through unified recurrent and convolutional neural networks,” Bioinformatics, vol. 35, no. 18, pp. 3329–3338, 2019.

[3] Z. Cui, X. Xu, X. Fei, X. Cai, Y. Cao, W. Zhang, and J. Chen, “Personalized recommendation system based on collaborative filtering for iot scenarios,” IEEE Transactions on Services Computing, vol. 13, no. 4, pp. 685–695, 2020.

[4] J. Davidson, B. Liebald, J. Liu, P. Nandy, T. Van Vleet, U. Gargi, S. Gupta, Y. He, M. Lambert, B. Livingston et al., “The youtube video recommendation system,” in Proceedings of the fourth ACM conference on Recommender systems, 2010, pp. 293–296.

[5] C. Gao, X. Wang, X. He, and Y. Li, “Graph neural networks for recommender sys­tem,” in Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, 2022, pp. 1623–1625.