探究擴散網路之學習演算法實現於類比積體電路之可行性

Exploring the feasibility of training Diffusion Network with on-chip circuitry

高子銘(Tzu-Ming Kao)

指導教授 ： 陳新

國立清華大學/電機資訊學院/電子工程研究所/碩士(2010年)

摘要

人們對於生物體內如何運作很感興趣，因此數學家利用數學方式建立類神經網路演算法，來模仿生物體內神經元之間互相連結和傳遞訊息，最後利用類神經網路得到代表學習訊號的參數，拿它來做訊號辨識或分類；而此論文所要探討的內容：探討擴散網路(Diffusion Network)[1]之學習演算法實現於類比積體電路之可行性。
首先是修改和簡化擴散網路之學習演算法，修改學習演算法是為了易於實現成類比積體電路，簡化學習演算法的目標是為了讓電路較好設計，而學習理論部份最重要的是應用蒙地卡羅期望值最大化(Monte Carlo Expectation Maximization)來得到代表待學習訊號的擴散網路參數，接著再利用最佳化方法修改學習演算法，經過修改後之學習演算法可利用模擬軟體(MATLAB)來驗證，藉由不同的訓練資料來找尋擴散網路參數的範圍和解析度，來了解擴散網路要成功學習訊號，對於精確度要求程度高低，這些前置模擬工作對於電路規格有很大的關連性；接著探討電路實現之可行性，最後再利用實驗室虛擬儀器工程平（Laboratory Virtual Instrumentation Engineering Workbench）來模擬擴散網路學習過程。

關鍵字

擴散網路 ； 積分器電路 ； 學習演算法 ； 最佳化方法

並列摘要

People have always been interested in how neurons work in organisms, and therefore mathmeaticans built neural network algorithms with the use of mathematical method. A neural network algorithm can mimic behaviors that connect and communicate messages between neurons in organisms. By using neural network, we got the learning signal’s representative parameters that can apply to recognize and classify different signals. The theme of the thesis is ‘exploring the feasibility of training diffusion network with on-chip circuitry’.
First, modify and simplify the training algorithm of Diffusion Network. The modification of training algorithm of the diffusion network is for an easier implementation in analog integrated circuit. On the other hand, simplifying the training algorithm of the diffusion network enables the design of analog integrated circuit to become easier. The most important part of training theory is getting the parameters of the training signal by using the algorithm, ‘Monte Carlo Expectation Maximization.’ The training algorithm is modified by optimization method and can be verified by the mathematic simulation software (MATLAB). Searching the range and resolution of the parameters in the different training signals helps us to understand how necessary the accuracy of parameter is for the diffusion network to be successful. The simulation work is closely related to the circuit specification, then the study explores the feasibility of training diffusion network learning processes by the Laboratory Virtual Instrumentation Engineering Workbench (LABVIEW).

並列關鍵字

diffusion network ； integrator circuit ； learning algorithm ； optimization method

參考文獻

[1] J. R. Movellan, P. Mineiro, and R. J. Williams, "A Monte Carlo EM approach for partially observable diffusion processes: Theory and applications to neural networks," Neural Computation, vol. 14, no. 7, pp. 1507-1544, 2002.

[2] H. Chen and A. F. Murray, "Continuous restricted Boltzmann machine with an implementable training algorithm," IEE Proceedings-Vision Image and Signal

Master's Thesis of National Tsing Hua University,Taiwan, 2007.

[4] Edwin K. P. Chong and Statislaw H. Zak , An Introduction to Optimization, 2nd Edition, 2001.

[5] G. Cauwenberghs, "An analog VLSI recurrent neural network learning a