液體狀態機中的非線性現象：隨機共振與臨界現象之研究

Liquid state machine (LSM)藉由模擬神經網路來達成機器學習中的分類工作。為了增進LSM的工作效率，我們針對LSM中的非線性性質進行進一步的研究。首先，我們研究一種被稱為隨機共振的非線性現象。雖然一般而言雜訊對於一個機器的效能是破壞性的，但在一些非線性系統內，卻可藉由加入雜訊來增加訊噪比。我們首先在LSM中觀察到隨機共振，並且觀察到一個現象：對同樣的一組資料，LSM在有雜訊的環境下進行學習的穩定性反而更高，這顯示雜訊的存在不但可以幫助訊號傳遞，還可以幫助神經網路進行學習。第二，我們研究LSM中的自組織臨界現象。許多人相信神經網路在臨界狀態中可以有更好的工作效率，因此研究臨界現象對於LSM效率的提升是重要的。與前人多使用power-law分布來決定臨界狀態不同，我們使用自回歸模型研究神經間的相關性來決定臨界狀態。我們發現在自回歸模型的特徵值最接近1的時候，LSM有著最佳的工作效率，說明可以使用這個方法來判斷神經網路的臨界特性。

關鍵字

機器學習；類神經網路；隨機共振；臨界現象；自回歸模型

並列摘要

Liquid state machine (LSM) is an artificial neural network that does classification task by simulating spiking neurons. In order to improve the performance of LSM, we analyze the nonlinear phenomena behind it. In this study, two topics are studied. First, we analyze the nonlinear effect called stochastic resonance, which describes the effect of noise. While noise is an unwelcome feature in most system, it is possible for noise to enhance the performance in a nonlinear system. We observe that stochastic resonance can occurs in LSM, and show that the existence of noise can also help a neural network to 'learn'. Second, we study the critical phenomenon in LSM. Since many people believe a neural network has best performance under critical state, it is an important issue to determine whether our LSM is in criticality. Usually, people use statistical method such as power-law to determine the criticality. However, we use auto-regressive model which estimate the dynamical correlation between neurons to find critical state. In this study, we will show the LSM has best performance where the eigenvalue distribution in auto-regressive model is closest to 1.

並列關鍵字

Machine learning ； Artificial neural network ； Stochastic resonance ； Critical phenomenon ； Auto-regressive model

參考文獻

2. Danielle Smith Bassett and ED Bullmore. Small-world brain networks. The neuroscientist, 12(6):512 523, 2006.

4. Henry Markram, Yun Wang, and Misha Tsodyks. Di erential signaling via the same axon of neocortical pyramidal neurons. Proceedings of the National Academy of Sciences, 95(9):5323 5328, 1998.

5. Misha V Tsodyks and Henry Markram. The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. Proceedings of the National Academy of Sciences, 94(2):719 723, 1997.

6. JJ Collins, Carson C Chow, Thomas T Imho , et al. Stochastic resonance without tuning. Nature, 376(6537):236 238, 1995.

7. Duncan J Watts and Steven H Strogatz. Collective dynamics of ‘small-world’networks. nature, 393(6684):440 442, 1998.

國際替代計量

液體狀態機中的非線性現象：隨機共振與臨界現象之研究

全文下載

主題瀏覽