硬體卷積神經網路之量化方法與模擬平台

Quantization Method and Simulation Framework for Hardware Convolution Neural Networks

林威宏(Wei-Hung Lin)

指導教授 ： 黃世旭

中原大學/電機資訊學院/電子工程研究所/碩士(2020年)

https://doi.org/10.6840/cycu202000568

摘要

本論文提出神經網路之硬體模擬平台，此平台可將硬體電路運算 套用至神經網路中，模擬神經網路硬體實現之精確度並權衡其硬體成 本。我們根據此平台模擬硬體實現之量化方法，並進一步提出量化分 析法 IL tuning 在低位元寬度下提高準確度。此外，我們提出兩種新的 低位元寬度量化方式並設計其硬體電路，一是結合動態定點(DFX)與 動態雙定點(DDFX)表示法的混合動態定點量化(HDFX)，二是使用非 對稱雙動態定點(ADFX)的非對稱混合動態定點量化(AHDFX)。我們 的模擬平台將近似電路與量化方法套用至神經網路並模擬其精確度， 根據實驗結果量化分析 IL tuning 可有效減少量化造成的誤差而提高 量化精確度。我們提出的混合動態定點量化與動態雙定點量化在相同 精確度下硬體成本更少;非對稱混合動態定點量化與動態雙定點量化 相比有更高的精確度，並且兩者都可實現 8bits 低位元寬度高精確量化。

關鍵字

卷積神經網路 ； 量化 ； 硬體模擬 ； 低位元寬度

並列摘要

In this thesis, we propose a hardware simulation framework for the neural network design. This framework can be used to simulate the hardware circuit behavior for the trade-off between accuracy and hardware cost. This framework can also be used to simulate the quantization method. We propose an integer length (IL) tuning method to analyze the integer length for maximizing the accuracy with low bit-width. Moreover, we propose two low-bit width quantization methods and design their corresponding hardware circuits, including hybrid dynamic fixed-point (HDFX) quantization that combines dynamic fixed-point and dynamic dual-fixed-point (DDFX) representations and asymmetric hybrid dynamic fixed-point (AHDFX) quantization that utilizes asymmetric DDFX. The framework applies approximate circuit and quantization method to neural network for simulating the accuracy. Experimental results show that IL tuning can improve accuracy and reduce errors caused by quantization. Experimental results also show that, under 8-bit width, compared with DDFX quantization, HDFX quantization has less hardware cost with the same accuracy, while AHDFX quantization has higher accuracy.

並列關鍵字

Convolution Neural Network ； Quantization ； Hardware Simulation ； Low-bit Width

參考文獻

[1] V. Sze, Y. Chen, T. Yang and J. S. Emer, "Efficient Processing of Deep Neural Networks: A Tutorial and Survey," in Proceedings of the IEEE, vol. 105, no. 12, pp. 2295-2329, 2017.

[2] M. Shah and R. Kapdi, "Object Detection using Deep Neural Networks," 2017 International Conference on Intelligent Computing and Control Systems (ICICCS), pp. 787-790, 2017.

[3] A. Krizhevsky, I. Sutskever, and G. Hinton. “Imagenet Classification with Deep Convolutional Neural Networks,” In Proceedings of the 25th International Conference on Neural Information Processing Systems, pp. 1097–1105, 2012.

[4] K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770778, 2016.

[5] S. Han, H. Mao, and W.J. Dally, “Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding.” International Conference on Learning Representations (ICLR), 2016.