- 學位論文
應用密度函數比對資料品質一致性之研究
Applying the density function to map the consistency of data quality of study
摘要
資料品質問題一直是引受關注的，從網路搜尋引擎可搜尋到約十億多筆的相關資訊網頁，可見資料品質在現實生活中已成為相當重要的討論議題。在許多實務應用上，不時會接觸到針對相同調查母體抽樣所得的兩個獨立資料庫。在沒有互相連結變數的情況下，就無法像關聯式資料庫利用連結變數，將所有資料串聯起來。因此，在比對變數間一致性時，就無法透過一對一的方式進行資料的對應。故本研究提出由觀察資料機率密度函數形態的角度，依據資料變數的屬性，分別從單一維度及多維度來尋找其適當機率分配函數，利用所估計的機率分配函數作為兩獨立資料間比對的基礎，計算出兩筆資料間的重疊係數，進而判定彼此資料間的一致、吻合程度，使得在變數使用上更 具可靠性。 根據本研究範例，對於產業創新與工商普查資料的實務上應用分析，建議利用不隨時間變動的屬質變數進行一致性比對，相對於屬量變數可得到較佳的比對結果。
並列摘要
The data quality problem has been focused. There are more than one billion related webpage from the internet search engine. Obviously, data quality has been become an important issue in real life. In many practical applications, one contacts two independent databases that sampling from the same investigative population. As without linking variable, that will not be able to merge overall data like relational database. Therefore, we are unable to map data consistency through one by one way. In this study, we observe a point of view with probability density function. According to the attribute of the variables, we find the appropriate one-dimension and multi-dimension probability distribution function. Then, we use the estimated probability distribution function to calculate the overlap coefficient between the similar variables of the two independent data. Finally, we will to judge the extent of data consistency and to cause the variable more reliable. From the practical analysis of industrial innovation survey and the industry commerce and service census data in this example of study, we suggest using the non-time-varying of discrete variables to carry on mapping data that will get better results than the continuous variables.