應用動態編碼於多頁面網頁之記錄邊界偵測與資訊擷取

Exploiting Dynamic Encoding and Multiple Pages for Record Boundary Detection and Data Extraction

陳明權(Ming-chuan Chen)

指導教授 ： 張嘉惠

國立中央大學/資訊電機學院/資訊工程學系/碩士(2014年)

摘要

記錄範圍偵測在擷取器推導(Wrapper Induction)中是一個很重要的環節，偵測的結果好壞會直接影響後續的排比以及最後的準確度。過去的方法多為在單一網頁中進行各個區塊相似度計算，擁有的資訊量較少，而使用樹狀結構的相似度計算也會造成計算量的上升。在本篇論文中我們參考來自同個網站的多個網頁，分析出各網頁中共同與差異的部分，克服單一網頁所缺乏的資訊；同時為減少多個網頁增加的計算量，系統分析的主要對象為DOM樹中的葉節點，其數量僅為所有節點的三成。藉由葉節點在多個網頁的分佈情形，本文提出動態編碼，對葉節點進行抽象化，用以突顯記錄的規律性，使得重複樣式探勘能得到較好的成效。最後對於記錄範圍的偵測，本文提出地標的概念，根據存在於各筆記錄中的地標，並藉由在樹狀結構中的走訪來推測相應的記錄範圍。在實驗與評估的部分，本篇論文使用了知名的資料集與過去幾個系統比較，皆能達到不錯的準確率。

關鍵字

記錄範圍偵測 ； 動態編碼 ； 資訊擷取

並列摘要

Record boundary detection plays an important role in wrapper induction and the quality of record boundary detection will affect the precision of alignment and extraction directly. Previous approaches usually focus on calculating similarity between blocksor measure tree similarity in a single page.
In this paper, we analyze multiple pages that are generated by the same website. By exploring common parts and different parts in pages, we can overcome the weakness in single-page approaches. Because the computation load will increase when we deal with more pages, the proposed approach only focus on leaf nodes in DOM tree, which are about 30 percent of all nodes. We propose dynamic encoding, which can abstract leaf nodes and emphasize the regularity of every data records. With the dynamic encoding, we reduce the numberof the repeated pattern discovered. Finally, we propose the idea of landmark, which is located in the data record, and detecting the record boundary by segmenting the DOM tree. In the experiment, we evaluate the efficiencyin our approach and compare the effectivenesswith other systems.

參考文獻

2. G.O. Arocena and A.O. Mendelzon, "WebOQL: restructuring documents, databases and Webs", Data Engineering, 1998. Proceedings., 14th International Conference on, 24-33, 1998.

6. C.H. Chang and S.C. Kuo, "OLERA: Semisupervised Web-Data Extraction with Visual Support", IEEE Intelligent Systems, Vol 19(6), pp.56-64, 2004

10. P. Gulhane, et al., "Exploiting content redundancy for web information extraction", Proc. VLDB Endow., Vol 3(1-2), pp.578-587, 2010

11. C.N. Hsu and M.T. Dung, "Generating finite-state transducers for semi-structured data extraction from the Web", Inf. Syst., Vol 23(9), pp.521-538, 1998

12. M. Kayed and C.H. Chang, "FiVaTech: Page-Level Web Data Extraction from Template Pages", Knowledge and Data Engineering, IEEE Transactions on, Vol 22(2), pp.249-263, 2010