Explicit Use of Term Occurrence Probabilities for Term Weighting in Text Categorization

Zafer Erenel ； Hakan Altinçay ； Ekrem Varoğlu

《Journal of Information Science and Engineering》 27卷3期 (2011/05) Pp. 819-834

https://doi.org/10.6688/JISE.2011.27.3.2

並列摘要

In this paper, the behaviors of leading symmetric and asymmetric term weighting schemes are analyzed in the context of text categorization. This analysis includes their weighting patterns in the two dimensional term occurrence probability space and the dynamic ranges of the generated weights. Additionally, one of the newly proposed term selection schemes, multi-class odds ratio, is considered as a potential symmetric weighting scheme. Based on the findings of this study, a novel symmetric weighting scheme derived as a function of term occurrence probabilities is proposed. The experiments conducted on Reuters-21578 ModApte Top10, WebKB, 7-Sectors and CSTR2009 datasets indicate that the proposed scheme outperforms other leading schemes in terms of macro-averaged and micro-averaged F1 scores.

並列關鍵字

text categorization ； supervised term weighting ； symmetric schemes ； term occurrence probabilities ； support vector machines