使用共軛梯度訓練法之前饋式類神經網路視窗程式設計

Windows Programming of Feedforward Neural Network by Conjugate Gradient Training Method

魯銘德(Ming-Te Lu)

指導教授 ： 林顯聖

國立臺北科技大學/工程學院/化學工程所/碩士(2006年)

https://doi.org/10.6841/NTUT.2006.00114

摘要

本研究室一直以開發一個免費且實用的前饋式類神經網路視窗程式為目標。以前版本的訓練方法為使用動量修正或可變學習速率之最陡坡度法，學習速率緩慢，到達收斂頗耗時間。

本研究的目的為使用共軛梯度法作為訓練網路參數的方法，希望能增快學習速率。且將以前版本數據點逐一訓練的方式改成批次訓練，結果顯示學習速率有顯著的改善。在訓練功能上，以前的版本都是到達設定的搜尋步數，程式才會停止，本研究另加兩項收斂條件，使程式達到收斂條件時會自動停止。並將訓練過程另以一視窗獨立顯示，讓使用者更清楚看到訓練的進度。

利用本研究所開發之前饋式類神經網路視窗程式，進行三個模擬範例的應用測試，及三種不同訓練方法的收斂速度之比較，且用本研究的程式與商業軟體Matlab對表面張力數據進行訓練比較，結果顯示本視窗程式有一定的實用性。

關鍵字

前饋式類神經網路 ； 視窗程式設計 ； 數據擬合 ； 共軛梯度法

並列摘要

Developing a free windows program of feedforward neural network is the object of our laboratory. The training methods in the former programs are the Momentum Back propagation algorithm or the Variable Learning Rate Back propagation algorithm. The training rate is slow and it takes much time to reach convergence.

The conjugate gradient training method is studied to speed up the training rate in this research. Unlike the conventional algorithms, the data were not trained one-by-one. The data were batch trained. It is shown that the convergence rate is speeded up. Two stop conditions are added to stop training procedure automatically. In addition, an independent training display window is used to shown the training procedure.

Three simulation cases were used to test the applicability of the developed program. The convergence rate of three training methods was studied. The data of surface tention were used to compares the training ability of this work with that of commertial software matlab. These results show that the program developed in this research is practicable.

並列關鍵字

Feedforward Neural Network ； Windows Programming ； Data-fitting ； Conjugate Gradient Method

參考文獻

[1] McCulloch, W.S., and Pitts, W., "A logical calculus of the ideas immanent in nervous activity,". Bull. Math. Biophys, vol. 5, pp115-133, 1943.

[2] Hebb, D.O., "The Organization of Behavior," Wiley, 1949.

[3] Rosenblatt, F., "The perceptron: a probabilistic model for information storage and organization in the brain," Psychol. rex, vol. 65, pp386-408, 1958.

[5] Minsky, M., and Papert, S.A., "Perceptrons: An introduction to Computational Geometry," MIT Press, 1969.

[6] Grossberg, S., "Classical and instrumental learning by neural networks," Prog. Theoret. Biol, vol. 3, pp51-141. 1974.