利用Hadoop分佈式計算結構提升醫療大型資料處理速度—以健保資料庫為例

Using Hadoop Distributed Computing Architecture to Enhance the Processing Speed of Large Medical Data– An Example of Taiwan Health Insurance Database

張卜仁(Pu-Jen Chang)

指導教授 ： 徐建業

臺北醫學大學/醫學科技學院/醫學資訊研究所/碩士(2014年)

https://doi.org/10.6831/TMU.2014.00104

摘要

目前國內推進醫療雲端化，醫學資料開始有所改變，如電子病歷、遠距離醫療資料，使得不同類型的大量資料隨之而來。在醫學研究中，常常使用到一些不同資料的串檔，亦或是在產業合作間，也需要不同類型資料庫的整合。本研究目的在於使用常用資料庫工具MS SQL、MySQL時，執行分析查詢及串檔大量資料，面臨暫存空間不足與資料處理時間過長的問題。本研究利用健保資料庫上的大量資料，執行查詢語法在Hadoop和MS SQL、MySQL的效能比較上，證實Hadoop應用在醫療大型資料庫處理資料時間的效能為佳。最後利用Hadoop系統與Web結合成一健保資料庫雲端資料分析系統，並有助於增進Hadoop在醫學資料分析上的應用。

關鍵字

Big Data、雲端、Hadoop

並列摘要

Currently promoting the cloud of health in Taiwan, medical data began to change, such as Electronic Medical Records, telemedicine data, so that different types of large amounts of data follow. We are often used to merge some of the different database or cooperation with other industries also needs to integrate different types of databases to do research in medical research. This study aimed to use common databases MS SQL、 MySQL, execute the query and analyze and connection Big Data, so face to problem is the temporary lack of space and process data time is too long. In this study, we used the Big Data on the Taiwan Health Insurance Database, execution search syntax in Hadoop and MS SQL、MySQL and confirmed Hadoop applications in the large medical databases time-consuming performance is better than the other. Finally, we use Hadoop systems and Web combined into a Taiwan Health Insurance Database Cloud Data Analysis Systems, and enhancing Hadoop applications in medical data analysis

並列關鍵字

Big Data, Cloud, Hadoop

參考文獻

association studies using Hadoop clusters. Bioinformatics. 2013 Jan1;Vol 29(1):pp.135-6. doi: 10.1093/bioinformatics/bts647. Epub 2012 Nov 29.

[6] Pratt B, Howbert JJ, Tasman NI, Nilsson EJ. MR-Tandem: parallel X!Tandem using Hadoop MapReduce on Amazon Web Services. Bioinformatics. 2012 Jan 1; Vol 28(1):pp.136-7. doi: 10.1093/bioinformatics/btr615. Epub 2011 Nov 8.

[29] Christina Hoffa , On the Use of Cloud Computing for Scientific Workflows,eScience, 2008. eScience '08. IEEE Fourth International Conference on , 7-12 Dec. 2008

[30] Dawei Jiang ,The Performance of MapReduce: An Indepth Study, Proceedings of the VLDB Endowment, Volume 3 Issue 1-2, September 2010

[31] Ronald C Taylor ,An overview of the Hadoop/MapReduce/HBase framework and its current applications in bioinformatics, BMC Bioinformatics 2010, 11(Suppl 12):S1