具自由回饋節點的高速硬體倒傳遞及回饋型類神經網路設計

Development of High Speed Hardware Based Back Propagation and Recurrent Neural Networks with Free Feedback Nodes

詹雅宇(Ya-Yu Jhan)

指導教授 ： 蔡孟伸

國立臺北科技大學/機電學院/自動化科技研究所/碩士(2011年)

摘要

類神經網路模仿生物神經網路的特性，由層狀排列的人工神經元相互連結成平行運算的計算系統。本論文在Quartus II開發軟體上編譯VHDL硬體描述語言，設計可在硬體上實現多種前饋式和回饋型類神經網路的數位邏輯電路。本論文採用環狀串列架構為基礎，設計三層及四層的倒傳遞類神經網路和回饋型類神經網路。環狀串列電路裡的運算單元同時接收匯流排上的資訊，經由運算單元裡的乘法累加器計算，模擬類神經網路前向運算的加權計算。活化函數以改良的片段線性法在硬體上實現，可供系統選擇的活化函數包含雙彎曲函數和雙曲線正切函數。類神經網路的回饋層根據管理單元的輸出訊號控制回饋節點的輸出，可以自由選擇回饋的節點和節點數。以管線的設計方式加速網路計算誤差，隱藏層誤差運算則採用硬體共構的方式設計，降低硬體設計需要的邏輯閘。透過分段計算技術，系統可以在固定的硬體資源下不用重新規劃整個電路，合成出不同大小的網路。本論文透過ModelSim驗證電路執行的功能，並可利用Nios II下達指令和網路架構參數。經由曲線擬合實驗，驗證本論文的架構可達到高效率及多種網路可供選擇的需求。

關鍵字

環狀串列架構 ； 倒傳遞類神經網路 ； 回饋型類神經網路 ； 自由回饋節點

並列摘要

Artificial Neural Networks（ANN）mimic the properties of biological neural network. Neurons are inter-connected and compose a parallel computing system architecture. This thesis implements the Artificial Neural Network（ANN）in hardware in Quartus II development environment. The proposed architecture performs forward and backward calculation during learning process for three or four-layer Back Propagation Artificial Neural Networks and Recurrent Neural Networks. The hardware implementation of the activation functions are based on improved Piece Wise Linear function（PWL）. The hardware-based activation function includes Sigmoid Function and Hyperbolic Tangent Function. The multiplexers control the switch of feedback nodes. Users are able to choose the nodes that provide feedback control. Backward calculation of output layer error is implemented in pipeline structure which improves the operating speed. In order to reduce the cost of hardware resources, the backward calculation of hidden layer error is implemented in Torodial structure which is also used during forward calculation. The proposed system is able to compose different neural networks with limited number of PEs by segmentation calculation. The two experimental results, curve fitting and predict problems, show that the system can reach higher performance and valid the results.

並列關鍵字

Toroidal Artificial Neural Network ； Back-Propagation Neural ； Piece-Wise Linear ； Recurrent Neural ； Free Feedback Nodes

參考文獻

[10] 葉彥智，具彈性架構的高速硬體倒傳遞及回饋型類神經網路設計，碩士論文，國立台北科技大學自動化科技研究所，台北，2010。

[1] S. Y. Kung, and J. N. Hwang, "A Unifying Algorithm /Architecture for Artificial Neural Networks," Proc. International Conference on Acoustics, Speed, and Signal Processing, Vol. 4, May 1989, pp. 2505-2508.

[2] D. Hammerstrom, "A VLSI Architecture for High-Performance, Low-Cost, On-Chip Learning," Proc. of International Joint Conference on Neural Networks, Jun. 1990, pp. 537-544.

[4] 吳靜宜，以FPGA開發手部動作辨識系統之類神經網路晶片，碩士論文，私立中原大學醫學工程學系，桃園，2004。

[6] 陳慶全，改良式環狀類神經架構之實現與應用，碩士論文，國立台北科技大學自動化科技研究所，台北，2006。