應用特徵空間距離訓練支撐向量機分類器

Applications of feature space distance measures in training support vector classifiers

吳國賓(Kuo-Ping Wu)

指導教授 ： 王勝德

國立臺灣大學/電機資訊學院/電機工程學研究所/博士(2008年)

https://doi.org/10.6342/NTU.2008.00060

摘要

決定支撐向量機的核心函數參數與誤差懲罰參數，在實用上是與待解問題非常相依的。網格搜尋是最常建議使用的方法。在訓練的過程中，各個參數組合用來訓練對應的分類器，其中效果最好的分類器及其參數將被保留使用。這個方法可以找到具有良好推論能力的分類器及其參數，然而訓練許多分類器將耗費大量時間。本論文提出，使用群組分離指標，以估計在不同特徵空間中，分類器的推論能力。該指標為在特徵空間中求得的群組內以及群組間距離，以及他們的組合。計算指標花費的時間，通常比訓練支撐向量機分類器少，因此可以快速選擇較佳的參數組合。實驗結果顯示，適當設計的指標，可以選到優良的參數組合，對應的分類器之偵測準確性，約與網格搜尋相當，然而訓練時間可以大幅減少。

關鍵字

支撐向量機 ； 特徵空間 ； 核心函數參數 ； 模式選擇 ； 群組內距離 ； 群組間距離

並列摘要

Determining the kernel and error penalty parameters for support vector machines (SVMs) is very problem-dependent in practice. The most popular method to decide the parameters is the grid search method. In the training process, classifiers are trained with different parameter combinations, and only one of the classifiers is required for the testing process. This method can find a parameter combination with good generalization ability, while it makes the training process time-consuming. In this thesis we propose using separation indexes to estimate the generalization ability of the classifiers. These indexes are derived from the inter- and intra-cluster distances in the feature spaces. Calculating such indexes often costs much less computation time than training the corresponding SVM classifiers; thus the proper parameters can be chosen much faster. Experiment results show that some of the indexes can choose proper kernel parameters with which the testing accuracy of trained SVMs is competitive to the standard ones, and the training time can be significantly shortened.

並列關鍵字

support vector machine ； feature space ； kernel function ； model selection ； intra-cluster distance ； inter-cluster distence

參考文獻

[1] V. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.

[2] C.-W. Hsu and C.-J. Lin, "A comparison of methods for multiclass support vector machines," IEEE trans. Neural Network, vol. 13, pp. 415-425, 2002.

[3] N. Ancona, G. Cicirelli, E. Stella, and A. Distante, "Object detection in images: run-time complexity and parameter selection of support vector machines," 16th International Conference on Pattern Recognition, vol. 2, pp. 426-429, 2002.

[4] S. Rojas and D. Fernandez-Reyes, "Adapting multiple kernel parameters for support vector machines using genetic algorithms," The 2005 IEEE Congress on Evolutionary Computation, vol. 1, pp. 626-631, 2005.

[5] A. T. Quang, Q.-L. Zhang, and X. Li, "Evolving support vector machine parameters," Proc. 2002 International Conference on Machine Learning and Cybernetics, vol. 1, pp. 548-551, 2002.