以neuro-fuzzy為基礎的Adaboost演算法

A neuro-fuzzy based on Adaboost algorithm

陳重光(Chong-Guang Chen)

指導教授 ： 秦群立

中山醫學大學/健康管理學院/醫學資訊學系碩士班/碩士(2016年)

摘要

過去十年，機器學習的發展，主要利用整合方法，其中利用了許多經過不斷調整的分類器來結合成一個高精確的分類器。本論文中，我們提出了一個整合神經模糊系統與Adaboost演算法的分類器。這就是所謂的混合神經模糊系統與Adaboost演算法之分類器。在分類器當中，Adaboost演算法會利用樣本的訓練，自適應性地迭代調整樣本權重，產生出多個分類器。而Adaboost的主要架構就是利用所產生出的多個分類器結合而成。另外在Adaboost弱分類器的挑選，我們所採用的是SONFIN。SONFIN是一種神經模糊系統，而且SONFIN具有on-line learning的能力。高維度的特徵，造成了程式的大量運算，為了降低程式運算的負擔，我們使用了特徵選擇的方法來降低時間複雜度，該方法利用基因搜尋(GeneticSearch)和排序搜尋(RankSearch)兩者的結合，結合的方式是先由基因搜尋挑選出最佳特徵，再利用排序搜尋依照挑選出來的特徵數量作增減，所選出的特徵在執行程式階段時，確實地成功降低程式運算。最後，為了證明我們所提出的分類器的能力，我們使用不同的資料集(Datasets)進行訓練以及測試，這些資料集包括了鳶尾花資料集(IRIS datasets)、威斯康辛乳房資料集(WISCONSIN breast datasets)和中山醫學大學乳房資料集(CSMU breast datasets)，這些資料集各自包含了不同的樣本數和特徵數，如此能夠讓系統獲得更好的準確率。本論文的貢獻為提出了混合神經模糊系統與Adaboost演算法之分類器，並且在IRIS datasets的分類精確度上達到98%準確率、在WISCONSIN breast datasets的分類精確度上達到99%準確率、在CSMU breast datasets的分類精確度上達到98.8%準確率。

關鍵字

神經模糊系統 ； Adaboost演算法 ； SONFIN ； 基因搜尋 ； 排序搜尋 ； 鳶尾花資料集 ； 威斯康辛乳房資料集 ； 中山醫學大學乳房資料集

並列摘要

One of the major developments in machine learning in the past decade is the Ensemble method, which finds a highly accurate classifier by combining many moderately accurate component classifiers. In this thesis, we propose a classifier integrated neuro-fuzzy system with adaboost algorithm. It is called Hybrid-neuro-fuzzy system and Adaboost-classifier. Herein, Adaboost creates a collection of component classifiers by maintaining a set of weights over training samples and adaptively adjusting these weights after each iteration, and it is main architecture. The weak learner in Adaboost we used is SONFIN which is a neuro-fuzzy system. And, there is on-line learning ability in SONFIN. The high dimensionality feature caused a large number of program operation. In order to reduce the loading of program implementation, we used the method of feature selection to reduce time complexity. The method utilizes the combination of GeneticSearch with RankSearch. The combined method is to start from picking out the best features from GeneticSearch, and using a number of features which selected from RankSearch to do increase or decrease. In the implementation phase of program, the selected features can surely successfully reduced implementation time. Finally, to demonstrate the capability of our proposed classifier, training and testing in different datasets including IRIS datasets, WISCONSIN breast datasets, and CSMU breast datasets are done. The contributions of this thesis include implementation of Hybrid-neuro-fuzzy system and Adaboost-classifier classifer for the classification and a classification accuracy of over 98% when training and testing on the IRIS dataset, 99% when training and testing on the WISCONSIN dataset, and 98.8% when training and testing on the CSMU Dataset.

並列關鍵字

Neuro-fuzzy system ； Adaboost algorithm ； SONFIN ； GeneticSearch ； RankSearch ； IRIS datasets ； WISCONSIN breast datasets ； CSMU breast datasets

參考文獻

[1] Schapire, R.E., Singer, Y., “Improved boosting algorithms using confidence-rated predictions,” Machine Learning, Vol. 37, pp. 297–336, 1999.

[2] Y. Freund and R. E. Schapire, “A decision-theoretic generalization of on-line learning and an application to boosting,” Journal of Computer and System Sciences, Vol. 55, pp. 119-139, 1997.

[3] Ming Yang, James Crenshaw, Bruce Augustine, Russell Mareachen and Ying Wu, “Adaboost-based face detection for embedded systems,” Computer Vision and Image Understanding, Vol. 114, pp.1116-1125, 2010.

[5] Qingfeng Nie, Lizuo Jin and Shumin Fei, “Probability estimation for multi-class classification using Adaboost,” Pattern Recognition, Vol. 47, pp.3931-3940, 2014.

[6] Jinna Lu, Hongping Hu and Yanping Bai, “Generalized radial basis function neural network based on an improved dynamic particle swarm optimization and Adaboost algorithm,” Neurocomputing, Vol. 152, pp.305-315, 2015.