以基於縱橫式可變電阻記憶體之類神經網路加速器的自適應資料表示法降低類比計算誤差

Adaptive Data Representation to Decrease Analog Variation Error of ReRAM Crossbar Accelerator for Neural Networks

康耀文(Yao-Wen Kang)

指導教授 ： 郭大維

共同指導教授 ： 張原豪(Yuan-Hao Chang)

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2019年)

https://doi.org/10.6342/NTU201900719

摘要

龐大的深度類神經網路運算導致密集的記憶體存取而因此限制了馮諾伊曼架構之效能。為了彌平這樣的效能落差，
以記憶體作為運算主體的架構被廣泛的提倡，在這些研究之中以縱橫式可變電阻記憶體加速器為一大主要解決方案。然而由於可變電阻記憶體之寫入偏差會導致此類加速器有著嚴重的準確度問題。為了改善此確度問題，我們提出了自適應資料表示法用以降低因為可變電阻記憶體之寫入偏差所導致的錯誤。我們基於真實的可變電阻記憶體晶片數據做了一系列實驗模擬。結果顯示出根據我們提出之方法可讓MNIST之辨識準確率提升20%及CIFAR10之辨識準確率提升40%

關鍵字

類神經網路 ； 可變電阻記憶體 ； 記憶體運算 ； 縱橫式

並列摘要

Current deep neural network computations incur intensive memory accesses and thus limit the performance of current Von-Neumann architecture. To bridge the performance gap, Processing-In-Memory (PIM) architecture is widely advocated and crossbar accelerators with Resistive Random-Access Memory (ReRAM) are one of the intensively-studied solutions. However, due to the programming variation of ReRAM, crossbar accelerators suffer from the serious accuracy issue. To improve the accuracy, we propose an adaptive data representation strategy to minimize the analog variation errors caused by the programming variation of ReRAM. The proposed strategy was evaluated by a series of intensive experiments based on the data collected from real ReRAM chips, and the results show that the proposed strategy can improve the accuracy for around 20% for MNIST which is close to the ideal case and 40% for CIFAR10.

並列關鍵字

Neural Network ； Resistive Random-Access Memory (ReRAM) ； Processing-In-Memory (PIM) ； Crossbar

參考文獻

[1] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processingsystems,pages1097–1105,2012.

[2] Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. Speech recognition with deep recurrent neural networks. In Acoustics, speech and signal processing (icassp),2013ieeeinternationalconferenceon,pages6645–6649.IEEE,2013.

[3] Ping Chi, Shuangchen Li, Cong Xu, Tao Zhang, Jishen Zhao, Yongpan Liu, Yu Wang, and Yuan Xie. Prime: A novel processing-in-memory architecture for neuralnetworkcomputationinreram-basedmainmemory.InACMSIGARCHComputerArchitectureNews,volume44,pages27–39.IEEEPress,2016.

[4] StephenWKeckler,WilliamJDally,BrucekKhailany,MichaelGarland,andDavid Glasco. Gpusandthefutureofparallelcomputing. IEEEMicro,(5):7–17,2011.

[5] H-SPhilipWong,Heng-YuanLee,ShimengYu,Yu-ShengChen,YiWu,Pang-Shiu Chen, Byoungil Lee, Frederick T Chen, and Ming-Jinn Tsai. Metal–oxide rram. ProceedingsoftheIEEE,100(6):1951–1970,2012.