同步分級神經網路在CUDA架構上的平行化研究

A Study on the Parallelism of Synchronous Ranked Neural Networks in CUDA System

陳星豪(Hsing-Hao Chen)

指導教授 ： 田伯隆

國立交通大學/電機學院/電信工程系所/碩士(2012年)

https://doi.org/10.6842/NCTU.2012.00327

摘要

在本篇論文中，我們將藉由NVDIA公司提出的一個利用圖形處理器(Graphic Processor Unit，GPU)的運算架構，名為計算統一設備架構(Compute Unified Device Architecture，CUDA)，模擬同步分級神經網路(Synchronous Ranked Neural-Network，SRNN)的平行分塊更新。另外，針對SRNN模型的運作特性，討論分塊更新及神經元等級分佈，對我們的SRNN模型最後達收斂態所需計算量造成的影響，希望最後能夠針對我們SRNN模型所要處理的問題，找出一些較佳的設定，使得我們能夠用較少的計算量，就讓我們的SRNN模型達收斂狀態。最後，我們利用分波多工光相位排列交換器互連系統(WDM OPAS-based Optical Interconnect System，WOPIS)內的封包排程問題，作為我們實作SRNN模型平行分塊更新的處理問題，並且從中驗證我們所提出的兩個影響因素，是否如我們所預料的方式，對收斂所需計算量造成影響。另外再針對CUDA在各個平行度下執行效率的觀察，找出一個更新區塊個數及平行度最理想的權衡比例。

關鍵字

同步分級神經網路

並列摘要

In this thesis, we using the compute unified device architecture (CUDA) which NVIDIA announced a graphic processor unit (GPU) computing architecture to simulate the feature that synchronous ranked neural network (SRNN) can be updated synchronously. And aim at the SRNN operating feature, we discuss the effect of block update manner and neurons’ rank distribution on the amount of computation of SRNN convergence. We hope that we can aim at different SRNN handling problems to find some better setting for minimum computation. In the end, we use the packet scheduling of WDM OPAS-based optical interconnect system (WOPIS) problem as our SRNN model’s handling problem in block update manner. And prove the two effect factors that is as our expectation. In addition, we also want to find the best trade-off between the number of update block and execution parallelism by observing the execution efficiency in different parallelism.

並列關鍵字

Synchronous Ranked Neural Network

參考文獻

[1] C. Lin and C. S. Lee, “Neural Fuzzy Systems A Neuro-Fuzzy Synergism to Intelligent Systems,” Prentice Hall 1996.

[2] X. Hu and B. Zhang, “A New Recurrent Neural Network for Solving Convex Quadratic Programming Problems With an Application to the k-Winners-Take-All Problem,” IEEE Trans. Neural Networks, vol. 20,no. 4, April 2009, pp. 654-664.

[3] P. Tien and B. Ke, “A new ranked Hopfield neural networks approach to QoS parallel scheduling for WDM optical interconnection system, ” IEEE High Performance Switching and Routing (HPSR), July 2011, pp. 276 - 281

[4] X. Gao and L. Liao, “A novel neural network for a class of convex quadratic minimax problems,” Neural Computing, vol. 18, no. 8, Aug. 2006, pp. 1818–1846.

[5] X. Hu and J. Wang, “A recurrent neural network for solving a class of general variational inequalities,” IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 37, no. 3, Jan. 2007, pp. 528–539.