具擴展性的高效能分散式圖資料處理系統

Kylin: An Efficient and Scalable Graph Data Processing System

李宗翰(Tsung-Han Li)

指導教授 ： 劉邦鋒

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2013年)

https://doi.org/10.6342/NTU.2013.01059

摘要

無資料

關鍵字

圖資料處理 ； 圖資料切割 ； 負載平衡 ； 動態載入

並列摘要

We introduce Kylin, an efficient and scalable graph data processing system. Kylin is based
on Bulk Synchronous Parallel (BSP) model to process graph data. Although there have
been some BSP-based graph processing systems, Kylin is different from these systems
in two-fold. First, Kylin cooperates with HBase to achieve scalable data manipulation.
Second, We propose three techniques to optimize the performance of Kylin. The proposed
techniques are pull messaging, lazy vertex loading and vertex-weighted partitioning. We
demonstrate Kylin outperforms other BSP-based systems, i.e. Hama and Giraph, in the
experiments.

並列關鍵字

graph data processing ； graph data partition ； load balancing ； pull messaging ； dynamic loading

參考文獻

Design and Implementation.

[2] Jeffrey Dean and Sanjay Ghemawat. Mapreduce: simplified data processing on large

[3] Leslie G. Valiant. A bridging model for parallel computation. Commun. ACM,

Horn, Naty Leiser, and Grzegorz Czajkowski. Pregel: a system for large-scale graph

processing. In Proceedings of the 2010 international conference on Management of