深度神經網路於現場可程式化邏輯閘陣列之高效實作與轉換方法

Mapping Deep Neural Network for Efficient FPGA Implementation

郭皓元(Hao-Yuan Kuo)

指導教授 ： 江介宏

國立臺灣大學/電機資訊學院/電子工程學研究所/碩士(2017年)

https://doi.org/10.6342/NTU201702277

摘要

隨著機器學習應用的蓬勃發展，深度神經網路的硬體實作是一個低功耗，高效能，且低面積的解決方案。實作上，為了減少高成本算數電路的使用，將神經網路作二元化的處理是一個很重要的步驟。現有的神經網路二元化方法，大致上需要依靠特殊的網路訓練方式或者利用隨機計算的技巧。給定網路拓樸架構的情況下，前者需要大量的實作經驗來找出適當的重新訓練步驟；而後者則是在不仰賴重新訓練網路的情形下，找出一個一般化且系統化的計算方法。在這篇論文中，透過上述的重新訓練以及隨機計算的方法，我們會探討如何將深度神經網路轉換並實作在現場可程式化邏輯閘陣列上。作為個案討論，我們將兩種訓練於MNIST手寫數字辨識資料集的深度神經網路實作在Xilinx Artix-7的現場可程式化邏輯閘陣列上，並且比較各實作方式之優劣。

關鍵字

機器學習 ； 神經網路 ； 乘法器 ； 隨機計算 ； 現場可程式化邏輯閘陣列

並列摘要

Deploying deep neural networks (DNNs) in hardware is a common tactic to achieve energy, performance, and area efficiency for widespread applications. Binarizing neural networks is a key step to eliminate costly arithmetic computation from circuit implementation.Available neuron binarization methods are based on either special training or stochastic computation.The former requires expertise and experience to find a workable training procedure for a given network architecture;the latter decouples the training process and is generally and systematically applicable to a well-trained neural network. In this thesis, we study how to map DNNs for efficient FPGA realization through retraining parameters and stochastic computation.
As a case study, a maxout DNN and ReLU DNN for MNIST dataset classification is trained and mapped onto Xilinx Artix-7 FPGA for efficient implementation.

並列關鍵字

machine learning ； neural network ； stochastic computing ； multipliers ； FPGA

參考文獻

[1] Cristanel Razafimandimby, Valeria Loscri, and Anna Maria Vegni. A neural network and iot based scheme for performance assessment in internet of robotic things. In Internet-of-Things Design and Implementation (IoTDI), 2016 IEEE First International Conference on, pages 241–246. IEEE, 2016.

[4] Andre Esteva, Brett Kuprel, Roberto A Novoa, Justin Ko, Susan M Swetter,Helen M Blau, and Sebastian Thrun. Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639):115–118, 2017.

[11] Ao Ren, Ji Li, Zhe Li, Caiwen Ding, Xuehai Qian, Qinru Qiu, Bo Yuan, and Yanzhi Wang. Sc-dcnn: highly-scalable deep convolutional neural network using

[12] Chen, Yu-Hsin and Krishna, Tushar and Emer, Joel and Sze, Vivienne. Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks. In IEEE International Solid-State Circuits Conference, ISSCC 2016,

[2] David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneer-shelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. Nature, 529(7587):484–489, 2016.