以聚合法(AGNES)提升檢索效果之研究—以中文新聞為例

The Research on Improving the Performance of Information Retrieval with the AGglomerative NESting (AGNES) Algorithm — Using a Chinese News Dataset

宋永杰(Yung-Chieh Sung)

指導教授 ： 魏世杰

淡江大學/管理學院/資訊管理學系碩士班/碩士(2007年)

https://doi.org/10.6846/TKU.2007.00927

摘要

傳統向量模式檢索系統回傳的相關資料往往過於雜亂缺乏系統，使用者必須花費心思逐步過濾，才能取得真正符合需求的資訊。本研究以聚合法所建構出的樹狀結構為基礎，由下而上動態群聚向量模式檢索系統所回傳的結果，形成多個群集，群集間依本研究之耦合力與內聚力的平均值做排名，群集內則依文章與查詢的相似度做排名，經調整排名後提升其精確率，並以群集的方式提供使用者瀏覽。
本研究採用中文文件集，經斷詞、特徵詞選取、建立文件向量、分群、檢索、群聚檢索結果與調整排名等處理。實驗結果顯示，在整體檢索表現中本系統可提升傳統向量模式檢索系統約20.9%~24.0%的精確率，經Wilcoxon Signed Ranks Test檢定，在1個關鍵詞與2個關鍵詞查詢下，本系統檢索表現優於傳統向量模式檢索系統。

關鍵字

資訊檢索 ； 聚合法 ； 分群 ； 向量空間模式

並列摘要

Usually the document ranking returned by the traditional vector space model of an information retrieval system is unorganized. It is often found that related documents do not have adjacent ranks. In order not to miss the needed information, the user still has to read several unrelated documents before finding another related document. In this research, we cluster the documents from the traditional vector space model based on the binary tree hierarchy constructed by the AGglomerative NESting (AGNES) algorithm. The clusters are ranked by the average of the coupling and the cohesion measures proposed in this thesis, and the documents in the cluster are ranked by the similarity between the query and the document. We try to improve the precision by such ranking adjustment.
We used the Chinese news dataset and went through the word segmentation, vector representation, AGNES clustering, query based document retrieval and the final ranking adjustments for evaluation. As result, our system can improve the precision by 20.9% to 24.0% compared to the traditional vector space model. We also tested the result by the Wilcoxon Signed Ranks Test. It shows that our system is significantly better than the traditional vector space model for queries of one or two keywords.

並列關鍵字

Information Retrieval ； Agglomerative Nesting Algorithm ； Clustering ； Vector Space Model

參考文獻

[4] J. Allan, Topic Detection and Tracking: Event-based information Organization, Kluwer Academic Publishers, 2002.

[9] H.C. Chang, C.C. Hsu, Y.W. Deng, “Unsupervised document clustering based on keyword clusters”, Proceedings of the International Symposium on Communications and Information Technologies 2004, pp. 1198-1203, October 2004.

[10] A. Chen, J. He, L. Xu, F. Gey and J. Meggs, “Chinese text retrieval without using a dictionary,” ACM SIGIR, pp. 42-49, 1997.

[11] H.H. Chen and J.C. Lee, “Identification and classification of proper names in Chinese texts,” Proceedings of 16th International Conference on Computational Linguistics, pp. 222-229, 1996.

[14] W. Frakes and R. Baeza-Yates, Information Retrieval: Data Structure & Algorithms, Prentice Hall PTR, 1992