利用簡化之Nelder-Mead與粒子群聚演算法 整合模型於類神經網路訓練

本論文提出簡化之Nelder-Mead與粒子群聚演算法整合模型於類神經網路訓練之研究，主要包括三個部分。在論文的第一部分，我們提出一個以Nelder-Mead 搜尋法與粒子群聚最佳化演算法為基礎的新式簡化整合性搜尋法，簡稱為SNM-PSO，並應用於訓練類神經網路。該演算法相較於其他相似整合性粒子群體最佳化演算法，更為簡單並且更著重於搜尋空間的探索。模擬結果顯示，我們提出的方法效能比其他相似整合性方法優越。在論文的第二部分，我們提出適應性最小截尾平方模糊神經網路以用於處理資料中的雜訊。在我們提出的方法中，一個重要的參數，截尾百分比可經由資料來自動決定。在數值範例中，梯度下降法、粒子群聚最佳化演算法和SNM-PSO將用於訓練所提出的模糊神經網路並且比較其結果。結果發現，適應性最小截尾平方模糊神經網路，較一般任意指定截尾百分比的最小截尾平方模糊神經網路，有較佳抑制離群值之強韌性。在論文的最後一部分，針對一般非線性迴歸問題，我們提出一個新的學習機：加法模型類神經網路。此學習機結合了類神經網路和半參數迴歸問題裡使用的加法模型。具體來說，此加法模型類神經網路是將加法模型嵌入於類神經網路的輸出層。本論文所提出的SNM-PSO將用於估測加法模型類神經網路的第一層參數。從一些範例中，我們比較一般類神經網路和我們所提出的加法模型類神經網路的模擬結果。結果顯示，在相似等量的參數下，加法模型類神經網路表現得比一般類神經網路好。

關鍵字

particle swarm optimization ； Nelder and Mead ； artificial neural network ； linear least trimmed squares estimators ； fuzzy neural network ； additive model

並列摘要

Neural network training using simplified hybrid Nelder and Mead (NM) and particle swarm optimization (PSO) are investigated in this dissertation, which consists of three parts. In the first part, a new and simplified hybrid algorithm mixing the simplex method of Nelder and Mead and particle swarm optimization algorithm, abbreviated as SNM-PSO, is proposed for the training of the artificial neural networks (ANNs). Our proposed method is simpler than other similar hybrid PSO methods and puts more emphasis on exploration of the search space. Some simulation results show that the performance of the proposed method outperforms that of other similar hybrid methods. In the second part of the dissertation, we propose the adaptive least trimmed squares fuzzy neural networks (ALTS-FNNs), which are generalizations of the linear least trimmed squares (LTS) estimators, to deal with data sets with outliers. In our method, the important parameter, i.e., the trimming percentage or trimming constant, is automatically determined by the data. The incremental gradient descent, PSO, and SNM-PSO algorithms are used to train the ALTS-FNNs for some numerical examples and performance comparisons are made. The proposed ALTS-FNNs are usually more robust against outliers in the data than the usual LTS-FNNs with arbitrarily specified trimming percentages. In the last part of the dissertation, we propose a new class of learning models, namely the additive artificial neural networks (AANNs), for general nonlinear regression problems. This class of learning machines combines the artificial neural networks and the additive models (AMs) frequently encountered in semiparametric regression problems. Specifically, the AMs are embedded in the output layer of the ANNs to form the AANNs. The proposed SNM-PSO method will be utilized for estimating the parameters in the first layer of the proposed AANNs. Several numerical examples are provided to compare the performances of the conventional ANNs and the proposed AANNs. In all these simulations, the AANNs are better than the conventional ANNs with the similar equivalent numbers of parameters.

並列關鍵字

粒子群聚演算法； Nelder-Mead單體法；類神經網路；最小截尾平方估測器；模糊神經網路；加法模型

參考文獻

[1] K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural Netw., vol. 2, no. 5, pp. 359–366, 1989.

[2] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning internal representations by error propagation,” Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1, pp. 318–362, 1986.

[3] E. J. Hartman, J. D. Keeler, and J. M. Kowalski, “Layered neural networks with Gaussian hidden units as universal approximations,” Neural Comput., vol. 2, no. 2, pp. 210–215, 1990.

[4] J. Park and I. W. Sandberg, “Universal approximation using radial basis function networks,” Neural Comput., vol. 3, no. 2, pp. 246–257, 1991.

[5] B. Kosko, “Fuzzy systems as universal approximators,” IEEE Trans. Comp., vol. 43, no. 11, pp. 1329–1333, 1994.

國際替代計量

利用簡化之Nelder-Mead與粒子群聚演算法整合模型於類神經網路訓練

全文下載

主題瀏覽