自動化關聯偵測SQL至HBase 動態轉換系統

Automatic Correlation Detection for SQL-to-HBase Dynamic Schema Transformation System

林學灝(XUE-HAO LIN)

指導教授 ： 賴槿峰

國立中正大學/工學院/資訊工程學系/碩士(2016年)

摘要

海量資料的時代來臨，傳統的關聯式資料庫管理系統在處理日益增加的資料效能不如以往，無法提供一些即時系統有效率的服務，用NoSQL資料庫來處理海量資料成為更佳的選擇，所以將SQL資料庫中的資料轉移到NoSQL資料庫的方法成為ㄧ大議題，其中Apache HBase是目前眾多NoSQL資料庫中被廣泛使用的資料庫。鑑此，本論文以MySQL和HBase進行實驗，並歸納三種類型的SQL至HBase資料庫Schema轉換的方法，此外我們還藉由感測SQL資料庫中不同的Table之間的關聯程度，來優化轉換後的Schema達到較佳的效能。

關鍵字

HBase ； SQL ； NoSQL ； 海量資料 ； 雲端運算

並列摘要

The age of Big Data is coming. With data sets increasing, traditional relational database management system has bad performance, and it has been unable to provide an efficient service to real-time system. NoSQL database become the better choice to handle massive data sets. Therefore, how to transform the existing data form SQL database to NoSQL database becoming an important issue. Apache HBase is the most popular NoSQL database and widely been used in many areas. Thus we experimented with MySQL and HBase in this paper and sum up three different types of SQL-to-HBase schema transformation method. We also optimize the performance of HBase by detecting the correlation between Tables in the SQL database.

並列關鍵字

HBase ； SQL ； NoSQL ； Big Data ； Cloud computing

參考文獻

[2] C.P. Chen, and C.-Y. Zhang, “Data-intensive applications, challenges, techniques and technologies: a survey on big data,” Information Sciences, vol. 275, pp. 314-347, 2014.

[3] S. Ghemawat, H. Gobioff, and S.-T. Leung, “The Google file system,” In Proceedings of 19th Symposium on Operating Systems Principles, pp. 29-43, 2003.

[9] D. Borthakur, J. S. Sarma, and J. Gary, “Apache Hadoop Goes Realtime at Facebook,” SIGMOD, pp. 1071-1080, 2011.

[11] B. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears, “Benchmarking Cloud Serving Systems with YCSB,” ACM Symposium on Cloud Computing (SoCC), 2010.

[14] R. Lawrence, “Integration and Virtualization of Relational SQL and NoSQL Systems including MySQL and MongoDB,” International Conference on Computational Science and Computational Intelligence, pp. 285-290, 2014.