以網頁識別及清理改善資料擷取的研究

Web page Classification and Cleaning for Information Extraction

劉仁宇(Jen-Yu Liu)

指導教授 ： 張嘉惠

國立中央大學/資訊電機學院/資訊工程學系碩士在職專班/碩士(2006年)

摘要

網際網路使用的普及，豐富資訊不斷量增下，使用者面臨最大難題不在於資訊內容的多寡，而在於擷取出的資料能否符合實際所需。在網頁內容擷取最常遇到兩項困難：一是目標區域外，會有一些無關的資料；在目標區域內，也會夾雜著少許雜訊，影響擷取的正確性；然而真正擷取的目標內容，卻也因字詞與字詞間沒有嚴謹的文法及界限，而無法完整識別。

基於此理由，本篇論文希藉由網頁清理技術來達成資料擷取的正確性。我們採用SVM分類器，配合頁面清理技術做為實際擷取的輸入網頁；另外在資料擷取上，採用SoftMealy擷取器，以Induction rule的演算法產出擷取規則。依據此種概念，提出CBIE（Cleaning Based Information Extraction）。我們的實驗從DBWorld中已確認Accepted paper公佈時程的各Conferences網站，辨識Accepted paper所在的網頁，再經由頁面清理擷取其中論文題目與作者，其結果顯示有相當程度改善效果，也證明頁面清理想法的可行性。

關鍵字

資料擷取 ； 機器學習

並列摘要

As the popularization of internet, one puzzle the users may be forced to face is not the large quantity of information, but the difficulty to extract the information they desired from the web pages. In web Information extraction, the researchers are confronted by at least two difficulties which may decrease the precision and accuracy of the results. The first is the irrelevant data that appears outside the target areas. The second is the noisy information garbled with desired contents inside the target areas. In addition to these, the desired contents may not be identified completely due to the lack of clear separator.

The purpose of this thesis is to solve those difficulties during web information extraction by incorporating page cleaning techniques. We use Support Vector Machine (SVM) to train a classifier for page cleaning. The cleaned pages are them applied to generated extraction rules by SoftMealy. The proposed idea, called CBIE(Cleaning Based Information Extraction), was applied on the extraction of paper titles and authors from accepted papers identified from websites the result shows that the cleaned pages were higher extractor performance them original web pages.

並列關鍵字

Information extraction ； machine learning

參考文獻

2. Chia-Hui Chang, Shao-Chen Lui, and Yen-Chin Wu. Applying pattern mining to Web information extraction. In Proceedings of the 5th Pacific Asia Conference on Knowledge Discovery and Data Mining (PAKDD-2000), pp. 4-16, Hong Kong, Apr 2001.

3. Chia-Hui Chang and Shao-Chen Lui. IEPAD: Information Extraction based on Pattern Discovery, In Proceedings of the 10th International Conference on World Wide Web (WWW10), pp. 595-609, Hong Kong, May 2001.

4. D. Embley, Y. Jiang, and Y.-K. Ng. Record-boundary discovery in web documents. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD’99), pages 467–478, Philadelphia, PA, 1999.

9. Chun-Nan Hsu and Ming-Tzung Dung. Generating finite-state transducers for semi-structured data. Journal of Information Systems, Special Issue on Semi-structured Data, Volume 23, pp. 521-537, Aug 1998.

10. Chun-Nan Hsu and Chien-Chi Chang. Finite-state transducers for semi-structured text mining. In Proceedings of IJCAI-99 Workshop on Text Mining: Foundations, Techniques and Applications, pp. 38-49, Stockholm, Sweden, 1999.