以資料庫查詢語言設計高效率資料轉置系統之研究

隨著企業的業務不斷持續發展，經濟全球化帶來的企業競爭持續加劇，越來越多的企業想要建構自己的資料庫整合系統，對於大量異質和分散的資料進行整合。期望從中獲得更多有用的資訊，以提升企業的經爭力。資料庫整合系統基本功能包含資料擷取、資料轉換、資料載入等三個關鍵部分。把不同的來源的資料、格式和特點的資料在進行一個有系統的整合，從而為企業或組織提供全方面的資料共享，這就是資料整合的理念。資料整合研究中最重要的就是對ETL方法的研究。本研究的重點是如何提升ETL開發效率。現有的ETL方法主要是透過傳統程式語言(ex:C、C++)來處理資料的轉換，這種方式造成開發與效能上的瓶頸。同時現有的ETL工具也並不適合複雜的ETL業務規則的開發。如何以資料庫查詢語言提高ETL開發的效率是本文研究的重點。目前企業數據環境是以關聯式資料庫為主要基本架構，而且資料庫系統的功能逐漸的強大，利用資料庫系統本身的功能就可以完成許多資料整合的過程。因此本研究利用資料庫系統強大的轉換功能，在關聯式資料庫的基礎上，提出了使用資料庫查詢語言為基本，實現ETL的轉換過程。該方法是用純粹的SQL語句和函數的方式來完成ETL過程，轉換的過程中，對資料庫直接進行操作。最後並且對SQL查詢語句的細節和執行上的效率進行改良，在一定的程度上減少了ETL過程的時間消耗。

關鍵字

ETL ； SQL ；資料庫

並列摘要

With the company's business continued sustainable development of economic globalization continued to intensify competition among enterprises, more and more companies want to build their own database analysis system for a large number of distributed and heterogeneous data integration. Expect to get more useful information to enhance the fight by force. Database analysis system basic functions including data extract, transformation, load in three key parts. The different sources of data, data formats and features of conducting a systematic integration, so as to provide all aspects of corporate or organizational information sharing, which is the data integration concept. Data integration is the most important studies on the ETL method. The focus of this study is how to improve ETL developer productivity. Existing methods mainly through traditional ETL programming language (ex: C, C + +) to handle data transformation, this means to cause the development and performance bottlenecks. While the existing ETL tools are not suitable for complex business rules ETL development. How to improve database query language ETL development efficiency is the focus of this study. Current enterprise data environment is a relational database as the primary infrastructure, and database system function gradually strong, the use of the database system itself functions you can perform many data integration process. Therefore, this study used the database system's powerful transformation function, in a relational database, based on the proposed use of a database query language as the basic realization of ETL transformation process. The method is to use pure SQL statements and functions the way to complete the ETL process, the transformation process, the right to operate the database directly. Finally and for the SQL on the details and implementation of efficiency improvements, to a certain extent reduce the ETL process time-consuming.

並列關鍵字

ETL ； SQL ； Database

參考文獻

[5] Surajit Chaudhuri. Automating Statistics Management for Query Optimizers,ACM SIGMOD,San Diego,USA,1999.

[6] Nicolasc Bruno,Surajit Chaudhuri. Exploiting Statistics on Query Expressions for Optimization,ACM SIGMOD,Washington,USA 2002.

[7] Alkis Simitsis,Panons Vassiliadis, etal. "Optimizimg ETL Process in Data Warehouses". Proc.21st IEEE Int’l Conf. Data Eng, 2005:1084-4627.

[11] Alkis Simitsisl,Panons Vassiliadis. A Methodology for the Conceptual Modeling of ETL Process.Mar. 2000.

[17] CCeri and Gottlob. "Translating SQL into reltational algebra: optimization,semantics,and equivalence of SQL queries". IEEE Transactions on Software Engineering,1985.11(4).

國際替代計量

以資料庫查詢語言設計高效率資料轉置系統之研究

全文下載

主題瀏覽