以FPGA實現二元神經網路之加速

Implementation for Binary Neural Network on FPGA

張博皓(Bo-Hao Chang)

指導教授 ： 李佩君

國立暨南國際大學/科技學院/電機工程學系/碩士(2021年)

https://doi.org/10.6837/ncnu202100284

摘要

本論文主要研究目的為解決神經網路 (Neural Network, NN) 佔用大量記憶體資源以及運算資源之問題以實現神經網路之加速。
為了解決神經網路佔用記憶體空間以及運算資源問題，本論文採用了神經網路二值化技術作為應用於現場可程式化邏輯閘陣列之神經網路加速器，二值化技術是將原神經網路模型以32 位浮點數之權重值 (weightings) 及啟動值 (activations) 參數量化至 1 位定點數形成二元神經網路 (Binary Neural Network, BNN)。32 位浮點數參數經過二值化後降低了神經網路模型所必需的記憶體空間，因為1 位定點數參數的特殊性，卷積運算 (convolution operation) 以及批正規化運算 (batch normalization operation) 可進行簡化以減少運算資源及加速推理。然而二元神經網路降低了特徵 (features) 的精度，因此二元神經網路導致嚴重的資訊損失。為了解決二元神經網路資訊損失的問題，本篇論文使用MeliusNet 所提出的二元神經網路加速器架構透過 Dense Connection 技術提升特徵圖 (feature map) 的容量 (capacity) 及使用Shortcut Connection 技術提升特徵圖的品質 (quality) 以減少資訊損失。
根據實驗結果，此二元神經網路加速器架構在資料集為Cifar10的訓練下能有約90%的辨識判斷準確率。

關鍵字

深度神經網路 ； 二元神經網路架構 ； Dense Connection ； Shortcut Connection ； FPGA

並列摘要

This thesis is in the purpose of solving the problem of the enormous demands of memory space and computational power in neural network (NN).
In order to reduce memory utilization and computation time in NN, this thesis adopts the binarization NN technology as NN accelerator on FPGA. The binarization technology quantizes the weightings and activations with stating in 32-bit floating point into 1-bit parameters as binary neural network (BNN). Since the binarization of weightings and activations, the demand of memory space in NN model can be reduced. However, the binarization of weightings and activations brings about the disadvantage of severe information loss. To solve this problem, this thesis utilizes the BNN accelerator architecture in MeliusNet to increase the capacity of the feature map through Dense Connection technology and to use Shortcut Connection technology to improve features of quality.
The experimental results shows that this binary neural network accelerator architecture can have about 90% accuracy in training with Cifar10 dataset.

並列關鍵字

Deep Neural Network ； Binary Neural Network ； Dense Connection ； Shortcut Connection ； FPGA

參考文獻

[1] J. Weng, N. Ahuja and T. S. Huang, "Cresceptron: a self-organizing neural network which grows adaptively," [Proceedings 1992] IJCNN International Joint Conference on Neural Networks, 1992, pp. 576-581 vol.1

[2] S. Walczak, "Artificial neural network medical decision support tool: predicting transfusion requirements of ER patients," in IEEE Transactions on Information Technology in Biomedicine, vol. 9, no. 3, pp. 468-474, Sept. 2005

[3] M. Navrátil, P. Dostálek and V. Křesálek, "Classification of audio sources using neural network applicable in security or military industry," 44th Annual 2010 IEEE International Carnahan Conference on Security Technology, 2010, pp. 369-374

[4] T. -A. Bui, P. -J. Lee, K. -Y. Lum, C. Loh and K. Tan, "Deep Learning for Landslide Recognition in Satellite Architecture," in IEEE Access, vol. 8, pp. 143665-143678, 2020

[5] Y. Cheng, F. X. Yu, R. S. Feris, S. Kumar, A. Choudhary and S. Chang, "An Exploration of Parameter Redundancy in Deep Networks with Circulant Projections," 2015 IEEE International Conference on Computer Vision (ICCV), 2015, pp. 2857-2865