強抗模糊神經網路深度學習之研究

Study on Deep Learning of Resistant Fuzzy Neural Networks

高豐譽(Feng-Yu Gao)

指導教授 ： 謝哲光 ； 鄭志宏

義守大學/智慧科技學院/電機工程學系/博士(2022年)

摘要

在現實世界中，手頭的數據總是難以避免地存在一部分離群值。這些離群點與大部分數據點位置較為分開，或者以某種方式偏離數據的一般模式。離群值的存在可能導致誤差的增加，並對統計估計產生較大的不良影響。在本論文的主要研究目標，是通過在所提出的模糊神經網路中引入強抗方法來減少離群值對於回歸預測函數或鑑別分類函數之影響。由此之故，本論文提出了具有強抗性的階層式模糊神經網路和樹狀模糊神經網路來增強模型對於離群值之強抗性。
為了解決由於規則或者輸入變數的數量增加而導致的維數詛咒問題，我們將使用階層式模糊神經網路並研究其對離群值的強抗性。每個階層式模糊神經網路的低維系統都是一個我們提出的擴充式模糊神經網路。本論文將利用輸出變數與輸入變數之相關性將輸入變數分組，並將每一組的輸入當成階層模糊系統之輸入。為了提高階層式模糊神經網路的強抗性，本研究採用了最小截尾平方誤差作為代價函數。此外，本研究結合了決策樹的優點提出了嶄新結構的模糊神經網路，即樹狀模糊神經網路。在此之前樹狀模糊神經網路的強抗性從未被討論過。最後，除了強抗回歸問題，我們也對強抗分類問題進行了研究。本研究提出了一種新的強健損失函數，即軟截尾分類交叉熵 (STCCE)。我們將使用三種不同的強健損失函數，即硬截尾分類交叉熵 (TCCE)、線性分類交叉熵 (LSCCE) 和STCCE，來研究深度模糊神經網路的強抗分類問題。
為了研究所提出的模型在回歸和分類問題上的強抗性，模擬的部分將使用真實數據集進行驗證，並在神經網路的輸出中加入不同種類的雜訊以模擬真實數據集。通過k-fold交叉驗證的驗證結果表明，並與目前先進的模型相比較，本論文提出的模型對於離群值更具有強抗性。

關鍵字

強抗回歸 ； 離群值 ； 模糊神經網路 ； 階層式模糊神經網路 ； 樹狀模糊神經網路

並列摘要

In real world, the available data very often contain some outliers. The outliers are usually separated from the majority of the data or do not follow the general trend of the data. The existence of outliers can lead to the increase of errors and a substantial adverse effect on the statistical estimates. In this study, our main goal is to downweight the influence exerted by the outliers by introducing resistant methods to fuzzy neural networks for both regression and classification problems. The resistant hierarchical fuzzy neural networks and the resistant tree-like fuzzy neural networks using resistant methods are proposed in this study.
In order to solve the “curse of dimensionality” problem caused by increasing the number of rules and/or input variables, we utilize hierarchical fuzzy neural networks (HFNNs) and study their resistance to outliers. Each low-dimensional system of a hierarchical fuzzy neural network is an augmented fuzzy neural network (AFNN). Correlations between the response and input predictors may be used to partition the whole input variables into several groups. The input variables in each group then become the inputs of a level in a HFNN. For the purpose of enhancing the resistance of the HFNNs, the least trimmed squared (LTS) criterion is set to be the cost function. Combing the advantage of decision trees, another structure, namely tree-like fuzzy neural networks (TL-FNNs) are proposed, which are a special kind of hierarchical fuzzy neural networks. As far as we know, the resistance of the TL-FNNs has never been discussed before. At last, in addition to resistant regression problems, resistant classification problems are studied. A new robust loss function is proposed, i.e., soft trimmed categorical cross-entropy (STCCE). We will examine the robust classification for multi-class classification problems for deep fuzzy neural networks using three different robust loss functions, namely the trimmed categorical cross-entropy (TCCE), linearly scored categorical cross-entropy (LSCCE), and STCCE.
Experiments on real-world datasets from public libraries will be performed to study the resistance of the proposed models both for regression and classification problems. Different kinds of noise are randomly added to the response of the neural networks. Cross validation results showed that, in comparison with the state-of-the-art models, our proposed models are more resistant to outliers.

並列關鍵字

Resistant regression ； Outlier ； Fuzzy neural network ； Hierarchical fuzzy neural network ； Tree-like fuzzy neural network

參考文獻

[1] G. P. McCabe and D. S. Moore, Introduction to the Practice of Statistics, 3rd ed. New York: Freeman & Company, 1999.

[2] D. M. Hawkins, Identification of Outliers. London: Chapman and Hall, 1980.

[3] H. Aguinis, R. K. Gottfredson, and H. Joo, “Best-practice recommendations for defining, identifying, and handling outliers,” Organizational Research Methods, vol. 16, no.2, pp. 270-301, 2013.

[4] J. W. Osborne and A. Overbay, “The power of outliers (and why researchers should always check for them),” Practical Assessment, Research, and Evaluation, vol. 9, no. 19, pp. 6, 2004.

[5] D. Ghosh and A. Vogt, “Outliers: An evaluation of methodologies,” in Proceedings of the Joint Statistical Meetings, San Diego, USA, 2012, vol. 2012, pp, 3455-1346.