使用準牛頓訓練法之前饋式類神經網路視窗程式設計

Windows Programming of Feedforward Neural Network by Quasi-Newton Training Method

林金貝(Chin-Pei Lin)

指導教授 ： 林顯聖

國立臺北科技大學/工程學院/化學工程研究所/碩士(2007年)

https://doi.org/10.6841/NTUT.2007.00161

摘要

近年來，類神經網路在工程上的應用開始備受重視，甚至在心理、統計和氣象學也被廣泛討論。本實驗室一直以開發簡易和實用的類神經網路視窗程式為目標，之前開發的版本，從使用動量修正最陡坡度法到共軛梯度法，收斂速度並不理想，故本研究改用準牛頓訓練法進行改善。
本研究使用Visual Basic2005 進行視窗程式開發，並以準牛頓法作為訓練網路參數的方法。程式中使用Davidon-Fletcher-Powell 方程式來求準Hessian矩陣，和以往相較，在結果顯示視窗增加收斂條件的即時運算，讓使用者可以更清楚程式停止的原因以及訓練的進度。
利用本研究所開發之前饋式類神經網路視窗程式，進行三個模擬範例的應用測試，並和之前本實驗室開發的訓練方法做比較，結果顯示本程式有一定的實用性且收歛速度也比較快。

關鍵字

前饋式類神經網路 ； 視窗程式設計 ； 數據擬合 ； 準牛頓法

並列摘要

Recently, Neural network has been applied to many engineering fields. People apply Neural network in psychology、 statistics and aerography. Developing a free and practical windows program of feedforward neural network is the object of our laboratory. The training methods used in the former studies are the momentum backpropagation method and Conjugate Gradient Method. However the training rate of those methods is slow and it must take much time to reach convergence. Quasi-Newton training method is studied to speed up the training rate in this research.
Visual Basic 2005 is used to develop the window-based program. Quasi-Newton method is used to train weights and biases of the neural network. In the program, Davidon-Fletcher-Powell method was used to calculate the quasi Hessian matrix. Some stop criterions are added so the programming can stop automatically.
Three simulation cases are used to test the applicability of the program. The convergence rate of three training methods was studied . These results show that the program is practicable and the convergence rate of Quasi-Newton methods is the fastest.

並列關鍵字

Feedforward Neural Network ； Windows Programming ； Data-fitting ； Quasi-Newton Method

參考文獻

文，國立臺北科技大學化學工程研究所，臺北，2005。

[1] W. S. McCulloch and W. Pitts, " A logical calculus of the ideas immanent in nervous activity, " Bull. Math Biophy, vol.5, 1943, pp. 115-133.

[2] D. O. Hebb, The Organization of Behavior, Wiley, 1949.

[3] F. Rosenblatt, "The perceptron: a probabilistic model for information storage and organization in the brain," Psychol. rex, vol. 65, 1958, pp. 386-408.

[5] M. M. Minsky and S. A. Papert, Perceptrons: An introduction to Computational Geometry, MIT Press, 1969.