強化學習輔助特徵選取

Using Reinforcement Learning in assisting feature selection

嚴大益(Da-Yi Yen)

指導教授 ： 林泓毅

國立臺中科技大學/資訊與流通學院/流通管理系碩士班/碩士(2020年)

摘要

傳統以向前或向後搜尋為基礎的特徵選取法都共同地遭遇到一個難題，就是m個最好的特徵並不是最佳的m個特徵(the m best features are not the best m features)，傳統的特徵選取法由於選取特徵是基於特徵集合所進行的，評估出來最佳的特徵一旦被選中之後便不會再更動，然而強化學習(reinforcement learning)不會掉入這個問題，透過學習環境的設計以及行動方案的指派，在每一次的學習迭代裡每一個狀態（亦即特徵）都會根據先前的學習經驗以及周遭特徵的探索改變選取策略。
本研究提出了一套以強化學習為搜尋基礎的特徵選取方法，此套方法基於特徵提取(feature fetching)以及特徵學習(feature learning)兩個階段，在特徵提取階段，先進行相關性分析(relevance analysis)運用Information Gain衡量資料集內所有輸入特徵特徵的分類效用，僅保留具有分類效用的特徵以建構強化學習中的學習環境。特徵學習階段，基於多維空間的概念建構學習環境，將可探索的方向定義為行動方案以及狀態之間獎勵值的計算是基於特徵之間的聯合效果亦即成效佳的組合具有較高的獎勵值。
為了驗證我們的方法，實驗使用七個大型生物醫學資料集，使用十倍交叉驗證降低資料所造成的偏差。我們藉由分類器的分類效能和區別能力兩個指表評估我們所提出的方法，實驗結果顯示，本文所提出的強化學習輔助特徵選取這套方法與向前搜尋為基礎的特徵選取方法MIFS、mRMR相較之下，均能在C4.5、SVM、NB以及k-NN分類器中獲得具競爭性的「分類效能」和「區別能力」，研究主要的貢獻在於運用我們的方法可以獲得搜尋效率的改善，以及可以獲得精簡的特徵組合。

關鍵字

強化學習 ； 特徵選取 ； 搜尋策略 ； 相關性分析 ； 冗餘分析 ； 資訊增益

並列摘要

Traditional feature selection methods based on forward or backward search always suffer from the problem. That is “the m best features are not the best m features. Once one feature is selected, its join cannot be changed. Reinforcement learning does not fall in this problem. Adopting the completely different strategy from traditional methods, the core of feature selection using reinforcement learning is according to the previous learning experiences and exploration of surrounding states.
In this study, we proposed a new feature selection scheme based on model-free reinforcement learning. This scheme comprises two stages. First is feature fetching and second is feature learning. In the feature fetching, Information Gain is used to measure the relevance degree between input features and the target class label. In stage of feature learning, we construct a learning environment based on spatial concept from two dimensions to nine dimensions. The directions in the underlying space are defined as selection actions. The reward value between states are evaluated by measure the combined effectiveness between two features. To verify our scheme, seven datasets are taken in our experiment as compared to the conventional feature selections (MIFS and mRMR). Classification accuracy and discrimination capability are explicitly evaluated for detailed analysis. The experimental results show our scheme not only performs the similar classification accuracy in C4.5, SVM, Naïve Bayes, and k-NN are considered, but also largely reduce the feature subset size.

並列關鍵字

reinforcement learning ； feature selection ； relevance analysis ； redundancy analysis ； information gain

參考文獻

Agnihotri, D., Verma, K.. & Tripathi, P. (2017). Variable global feature selection scheme for automatic classification of text documents. Expert Systems with Applications(81 ), pp. 268–281.

Al Janabi, K. B.S. & Kadhim R. (2018, 11). Data reduction techniques: a comparative study for attribute selection method. international journal of advanced computer science and applications, pp. 1-13.

Battiti, R. (1994, 6). Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks, 5(4), pp. 537 - 550.

Bennasar, M. , Hicks, Y. & Setchi, R. (2015). Feature selection using joint mutual information maximisation. Expert Systems with Applications, 42(22), pp. 8520–8532.

Berry, D. A. & Fristedt, B. (1985). Bandit Problems: Sequential Allocation of Experiments. Chapman and Hall.